[PATCH v5 0/2] man-pages: document reflink/dedupe ioctls

[PATCH v5 0/2] man-pages: document reflink/dedupe ioctls

[Darrick J. Wong]

Hi all,

This patch set goes along with the fifth revision of an RFC adding to
XFS kernel support for tracking reverse-mappings of physical blocks to
file and metadata; and support for mapping multiple file logical
blocks to the same physical block, more commonly known as reflinking.

The two patches in this series document the clone, clone_range, and
dedupe ioctls in the form of man pages.

The patch set is based on a whole pile of proposed patches against the
current (4.5) upstream kernel; the ioctls are simply hoisted version
of the private btrfs ioctls.

Comments and questions are, as always, welcome.

--D

[PATCH 1/2] man2: document FICLONE and FICLONERANGE

[Darrick J. Wong]

Document the FICLONE and FICLONERANGE ioctls, formerly known as the
BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 man2/ioctl_ficlone.2      |    1 
 man2/ioctl_ficlonerange.2 |  124 +++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 125 insertions(+)
 create mode 100644 man2/ioctl_ficlone.2
 create mode 100644 man2/ioctl_ficlonerange.2


diff --git a/man2/ioctl_ficlone.2 b/man2/ioctl_ficlone.2
new file mode 100644
index 0000000..19bb348
--- /dev/null
+++ b/man2/ioctl_ficlone.2
@@ -0,0 +1 @@
+.so man2/ioctl_ficlonerange.2
diff --git a/man2/ioctl_ficlonerange.2 b/man2/ioctl_ficlonerange.2
new file mode 100644
index 0000000..c5c72aa
--- /dev/null
+++ b/man2/ioctl_ficlonerange.2
@@ -0,0 +1,124 @@
+.\" Copyright (C) 2016 Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(VERBATIM)
+.\" This program is free software; you can redistribute it and/or
+.\" modify it under the terms of the GNU General Public License as
+.\" published by the Free Software Foundation.
+.\"
+.\" This program is distributed in the hope that it would be useful,
+.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
+.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+.\" GNU General Public License for more details.
+.\"
+.\" You should have received a copy of the GNU General Public License
+.\" along with this program; if not, write the Free Software Foundation,
+.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+.\" %%%LICENSE_END
+.TH IOCTL-FICLONERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
+.SH NAME
+ioctl_ficlonerange, ioctl_ficlone \- share some the data of one file with another file
+.SH SYNOPSIS
+.br
+.B #include <sys/ioctl.h>
+.br
+.B #include <linux/fs.h>
+.sp
+.BI "int ioctl(int " dest_fd ", FICLONERANGE, struct file_clone_range * " arg );
+.br
+.BI "int ioctl(int " dest_fd ", FICLONE, int " src_fd );
+.SH DESCRIPTION
+If a filesystem supports files sharing physical storage between multiple
+files ("reflink"), this
+.BR ioctl (2)
+system call can be used to make some of the data in the
+.B src_fd
+file appear in the
+.B dest_fd
+file by sharing the underlying storage, which is faster than making a separate
+physical copy of the data.  If a file write should occur to a shared region,
+the filesystem must ensure that the changes remain private to the file being
+written.  This behavior is commonly referred to as "copy on write".
+
+This ioctl reflinks up to
+.IR src_length
+bytes from file descriptor
+.IR src_fd
+at offset
+.IR src_offset
+into the file
+.IR dest_fd
+at offset
+.IR dest_offset ",
+provided that both are files.  This information is conveyed in a structure of
+the following form:
+.in +4n
+.nf
+
+struct file_clone_range {
+        __s64 src_fd;
+        __u64 src_offset;
+        __u64 src_length;
+        __u64 dest_offset;
+};
+
+.fi
+.in
+Clones are atomic with regards to concurrent writes, so no locks need to be
+taken to obtain a consistent cloned copy.
+
+The FICLONE ioctl clones entire files.
+.SH RETURN VALUE
+On error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B EXDEV
+.IR dest_fd " and " src_fd
+are not on the same mounted filesystem.
+.TP
+.B EISDIR
+One of the files is a directory and the filesystem does not support shared
+regions in directories.
+.TP
+.B EINVAL
+The filesystem does not support reflinking the ranges of the given files.  This
+error can also appear if either file descriptor represents a device, fifo, or
+socket.  Disk filesystems generally require the offset and length arguments
+to be aligned to the fundamental block size.  XFS and btrfs do not support
+overlapping reflink ranges in the same file.
+.TP
+.B EBADF
+.IR src_fd
+is not open for reading;
+.IR dest_fd
+is not open for writing or is open for append-only writes; or the filesystem
+which
+.IR src_fd
+resides on does not support reflink.
+.TP
+.B EPERM
+.IR dest_fd
+is immutable.
+.TP
+.B ETXTBSY
+One of the files is a swap file.  Swap files cannot share storage.
+.TP
+.B EOPNOTSUPP
+This can appear if the filesystem does not support reflinking either file
+descriptor.
+.SH NOTES
+Because a copy on write operation requires the allocation of new storage, the
+.B fallocate (2)
+operation may un-share shared blocks to guarantee that subsequent writes will
+not fail because of lack of disk space.
+.SH CONFORMING TO
+This API is Linux-specific.  This ioctl was previously known as
+.B BTRFS_IOC_CLONE_RANGE
+and was private to btrfs.
+.fi
+.in
+.SH SEE ALSO
+.BR ioctl (2)

[PATCH 2/2] man2: document the FIDEDUPERANGE ioctl

[Darrick J. Wong]

Document the FIDEDUPERANGE ioctl, formerly known as BTRFS_IOC_EXTENT_SAME.

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
---
 man2/ioctl_fideduperange.2 |  168 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 168 insertions(+)
 create mode 100644 man2/ioctl_fideduperange.2


diff --git a/man2/ioctl_fideduperange.2 b/man2/ioctl_fideduperange.2
new file mode 100644
index 0000000..0fbc388
--- /dev/null
+++ b/man2/ioctl_fideduperange.2
@@ -0,0 +1,168 @@
+.\" Copyright (C) 2016 Oracle.  All rights reserved.
+.\"
+.\" %%%LICENSE_START(VERBATIM)
+.\" This program is free software; you can redistribute it and/or
+.\" modify it under the terms of the GNU General Public License as
+.\" published by the Free Software Foundation.
+.\"
+.\" This program is distributed in the hope that it would be useful,
+.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
+.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+.\" GNU General Public License for more details.
+.\"
+.\" You should have received a copy of the GNU General Public License
+.\" along with this program; if not, write the Free Software Foundation,
+.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+.\" %%%LICENSE_END
+.TH IOCTL-FIDEDUPERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
+.SH NAME
+ioctl_fideduperange \- share some the data of one file with another file
+.SH SYNOPSIS
+.br
+.B #include <sys/ioctl.h>
+.br
+.B #include <linux/fs.h>
+.sp
+.BI "int ioctl(int " src_fd ", FIDEDUPERANGE, struct file_dedupe_range * " arg );
+.SH DESCRIPTION
+If a filesystem supports files sharing physical storage between multiple
+files, this
+.BR ioctl (2)
+system call can be used to make some of the data in the
+.B src_fd
+file appear in the
+.B dest_fd
+file by sharing the underlying storage if the file data is identical
+("deduplication").  This reduces storage consumption by allowing the filesystem
+to store one shared copy of the data.  If a file write should occur to a shared
+region, the filesystem must ensure that the changes remain private to the file
+being written.  This behavior is commonly referred to as "copy on write".
+
+This ioctl performs the "compare and share if identical" operation on up to
+.IR src_length
+bytes from file descriptor
+.IR src_fd
+at offset
+.IR src_offset ".
+This information is conveyed in a structure of the following form:
+.in +4n
+.nf
+
+struct file_dedupe_range {
+        __u64 src_offset;
+        __u64 src_length;
+        __u16 dest_count;
+        __u16 reserved1;
+        __u32 reserved2;
+        struct file_dedupe_range_info info[0];
+};
+.fi
+.in
+Deduplication is atomic with regards to concurrent writes, so no locks need to
+be taken to obtain a consistent deduplicated copy.
+
+The fields 
+.IR reserved1 " and " reserved2
+must be zero.
+
+Destinations for the deduplication operation are conveyed in the array at the
+end of the structure.  The number of destinations is given in
+.IR dest_count ",
+and the destination information is conveyed in the following form:
+
+.in +4n
+.nf
+struct file_dedupe_range_info {
+        __s64 dest_fd;
+        __u64 dest_offset;
+        __u64 bytes_deduped;
+        __s32 status;
+        __u32 reserved;
+};
+
+.fi
+.in
+
+Each deduplication operation targets
+.IR length
+bytes in file descriptor
+.IR dest_fd
+at offset
+.IR logical_offset ".
+The field
+.IR reserved
+must be zero.
+
+Upon successful completion of this ioctl, the number of bytes successfully
+deduplicated is returned in
+.IR bytes_deduped
+and a status code for the deduplication operation is returned in
+.IR status ".
+
+The
+.IR status
+code is set to
+.B 0
+for success, a negative error code in case of error, or
+.B FILE_DEDUPE_RANGE_DIFFERS
+if the data did not match.
+
+.SH RETURN VALUE
+On error, \-1 is returned, and
+.I errno
+is set to indicate the error.
+.PP
+.SH ERRORS
+Error codes can be one of, but are not limited to, the following:
+.TP
+.B EXDEV
+.IR dest_fd " and " src_fd
+are not on the same mounted filesystem.
+.TP
+.B EISDIR
+One of the files is a directory and the filesystem does not support shared
+regions in directories.
+.TP
+.B EINVAL
+The filesystem does not support deduplicating the ranges of the given files.
+This error can also appear if either file descriptor represents a device, fifo,
+or socket.  Disk filesystems generally require the offset and length arguments
+to be aligned to the fundamental block size.  Neither btrfs nor XFS support
+overlapping deduplication ranges in the same file.
+.TP
+.B EBADF
+.IR src_fd
+is not open for reading;
+.IR dest_fd
+is not open for writing or is open for append-only writes; or the filesystem
+which
+.IR src_fd
+resides on does not support deduplication.
+.TP
+.B EPERM
+.IR dest_fd
+is immutable.
+.TP
+.B ETXTBSY
+One of the files is a swap file.  Swap files cannot share storage.
+.TP
+.B EOPNOTSUPP
+This can appear if the filesystem does not support deduplicating either file
+descriptor.
+.SH NOTES
+Because a copy on write operation requires the allocation of new storage, the
+.B fallocate (2)
+operation may un-share shared blocks to guarantee that subsequent writes will
+not fail because of lack of disk space.
+
+Some filesystems may limit the amount of data that can be deduplicated in a
+single call.
+
+.SH CONFORMING TO
+This API is Linux-specific.  This ioctl was previously known as
+.B BTRFS_IOC_FILE_EXTENT_SAME
+and was private to btrfs.
+.fi
+.in
+.SH SEE ALSO
+.BR ioctl (2)

Re: [PATCH v5 0/2] man-pages: document reflink/dedupe ioctls

[Christoph Hellwig]

Both man pages looks fine to me:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH 1/2] man2: document FICLONE and FICLONERANGE

[walter harms]

hi,
i tried to understand the man page and had some problems
(maybe because i am not an FS expert)

So far i understand the use case is to make a virtual copy
of a file (or range). but it works only on the same fs.

reading this:
https://lwn.net/Articles/331576/

I got the impression there is already a syscall ?
Since when is this function available ?

Perhaps you can add a simple use case so readers get a better
idea how/why to use this ?

just my 2 cents,
re,
 wh


Am 15.03.2016 17:48, schrieb Darrick J. Wong:
> Document the FICLONE and FICLONERANGE ioctls, formerly known as the
> BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  man2/ioctl_ficlone.2      |    1 
>  man2/ioctl_ficlonerange.2 |  124 +++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 125 insertions(+)
>  create mode 100644 man2/ioctl_ficlone.2
>  create mode 100644 man2/ioctl_ficlonerange.2
> 
> 
> diff --git a/man2/ioctl_ficlone.2 b/man2/ioctl_ficlone.2
> new file mode 100644
> index 0000000..19bb348
> --- /dev/null
> +++ b/man2/ioctl_ficlone.2
> @@ -0,0 +1 @@
> +.so man2/ioctl_ficlonerange.2
> diff --git a/man2/ioctl_ficlonerange.2 b/man2/ioctl_ficlonerange.2
> new file mode 100644
> index 0000000..c5c72aa
> --- /dev/null
> +++ b/man2/ioctl_ficlonerange.2
> @@ -0,0 +1,124 @@
> +.\" Copyright (C) 2016 Oracle.  All rights reserved.
> +.\"
> +.\" %%%LICENSE_START(VERBATIM)
> +.\" This program is free software; you can redistribute it and/or
> +.\" modify it under the terms of the GNU General Public License as
> +.\" published by the Free Software Foundation.
> +.\"
> +.\" This program is distributed in the hope that it would be useful,
> +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
> +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +.\" GNU General Public License for more details.
> +.\"
> +.\" You should have received a copy of the GNU General Public License
> +.\" along with this program; if not, write the Free Software Foundation,
> +.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +.\" %%%LICENSE_END
> +.TH IOCTL-FICLONERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
> +.SH NAME
> +ioctl_ficlonerange, ioctl_ficlone \- share some the data of one file with another file
> +.SH SYNOPSIS
> +.br
> +.B #include <sys/ioctl.h>
> +.br
> +.B #include <linux/fs.h>
> +.sp
> +.BI "int ioctl(int " dest_fd ", FICLONERANGE, struct file_clone_range * " arg );
> +.br
> +.BI "int ioctl(int " dest_fd ", FICLONE, int " src_fd );
> +.SH DESCRIPTION
> +If a filesystem supports files sharing physical storage between multiple
> +files ("reflink"), this
> +.BR ioctl (2)
> +system call can be used to make some of the data in the
> +.B src_fd
> +file appear in the
> +.B dest_fd
> +file by sharing the underlying storage, which is faster than making a separate
> +physical copy of the data.  If a file write should occur to a shared region,
> +the filesystem must ensure that the changes remain private to the file being
> +written.  This behavior is commonly referred to as "copy on write".
> +
> +This ioctl reflinks up to
> +.IR src_length
> +bytes from file descriptor
> +.IR src_fd
> +at offset
> +.IR src_offset
> +into the file
> +.IR dest_fd
> +at offset
> +.IR dest_offset ",
> +provided that both are files.  This information is conveyed in a structure of
> +the following form:
> +.in +4n
> +.nf
> +
> +struct file_clone_range {
> +        __s64 src_fd;
> +        __u64 src_offset;
> +        __u64 src_length;
> +        __u64 dest_offset;
> +};
> +
> +.fi
> +.in
> +Clones are atomic with regards to concurrent writes, so no locks need to be
> +taken to obtain a consistent cloned copy.
> +
> +The FICLONE ioctl clones entire files.
> +.SH RETURN VALUE
> +On error, \-1 is returned, and
> +.I errno
> +is set to indicate the error.
> +.PP
> +.SH ERRORS
> +Error codes can be one of, but are not limited to, the following:
> +.TP
> +.B EXDEV
> +.IR dest_fd " and " src_fd
> +are not on the same mounted filesystem.
> +.TP
> +.B EISDIR
> +One of the files is a directory and the filesystem does not support shared
> +regions in directories.
> +.TP
> +.B EINVAL
> +The filesystem does not support reflinking the ranges of the given files.  This
> +error can also appear if either file descriptor represents a device, fifo, or
> +socket.  Disk filesystems generally require the offset and length arguments
> +to be aligned to the fundamental block size.  XFS and btrfs do not support
> +overlapping reflink ranges in the same file.
> +.TP
> +.B EBADF
> +.IR src_fd
> +is not open for reading;
> +.IR dest_fd
> +is not open for writing or is open for append-only writes; or the filesystem
> +which
> +.IR src_fd
> +resides on does not support reflink.
> +.TP
> +.B EPERM
> +.IR dest_fd
> +is immutable.
> +.TP
> +.B ETXTBSY
> +One of the files is a swap file.  Swap files cannot share storage.
> +.TP
> +.B EOPNOTSUPP
> +This can appear if the filesystem does not support reflinking either file
> +descriptor.
> +.SH NOTES
> +Because a copy on write operation requires the allocation of new storage, the
> +.B fallocate (2)
> +operation may un-share shared blocks to guarantee that subsequent writes will
> +not fail because of lack of disk space.
> +.SH CONFORMING TO
> +This API is Linux-specific.  This ioctl was previously known as
> +.B BTRFS_IOC_CLONE_RANGE
> +and was private to btrfs.
> +.fi
> +.in
> +.SH SEE ALSO
> +.BR ioctl (2)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-man" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] man2: document FICLONE and FICLONERANGE

[Darrick J. Wong]

On Sat, Mar 19, 2016 at 06:04:54PM +0100, walter harms wrote:
> hi,
> i tried to understand the man page and had some problems
> (maybe because i am not an FS expert)
> 
> So far i understand the use case is to make a virtual copy
> of a file (or range). but it works only on the same fs.

Correct, these ioctls allow files within a filesystem to share some
of the blocks being managed by that filesystem.

I will make that clearer in the manpage.

> reading this:
> https://lwn.net/Articles/331576/
> 
> I got the impression there is already a syscall ?
> Since when is this function available ?

It isn't; the reflink syscall became an ocfs2 ioctl and never progressed
beyond that.

> Perhaps you can add a simple use case so readers get a better
> idea how/why to use this ?

cp --reflink is the best known client of this interface, though I've not
mentioned this in the manpage in case they should decide some day to use
something else.

--D
> 
> just my 2 cents,
> re,
>  wh
> 
> 
> Am 15.03.2016 17:48, schrieb Darrick J. Wong:
> > Document the FICLONE and FICLONERANGE ioctls, formerly known as the
> > BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  man2/ioctl_ficlone.2      |    1 
> >  man2/ioctl_ficlonerange.2 |  124 +++++++++++++++++++++++++++++++++++++++++++++
> >  2 files changed, 125 insertions(+)
> >  create mode 100644 man2/ioctl_ficlone.2
> >  create mode 100644 man2/ioctl_ficlonerange.2
> > 
> > 
> > diff --git a/man2/ioctl_ficlone.2 b/man2/ioctl_ficlone.2
> > new file mode 100644
> > index 0000000..19bb348
> > --- /dev/null
> > +++ b/man2/ioctl_ficlone.2
> > @@ -0,0 +1 @@
> > +.so man2/ioctl_ficlonerange.2
> > diff --git a/man2/ioctl_ficlonerange.2 b/man2/ioctl_ficlonerange.2
> > new file mode 100644
> > index 0000000..c5c72aa
> > --- /dev/null
> > +++ b/man2/ioctl_ficlonerange.2
> > @@ -0,0 +1,124 @@
> > +.\" Copyright (C) 2016 Oracle.  All rights reserved.
> > +.\"
> > +.\" %%%LICENSE_START(VERBATIM)
> > +.\" This program is free software; you can redistribute it and/or
> > +.\" modify it under the terms of the GNU General Public License as
> > +.\" published by the Free Software Foundation.
> > +.\"
> > +.\" This program is distributed in the hope that it would be useful,
> > +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +.\" GNU General Public License for more details.
> > +.\"
> > +.\" You should have received a copy of the GNU General Public License
> > +.\" along with this program; if not, write the Free Software Foundation,
> > +.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> > +.\" %%%LICENSE_END
> > +.TH IOCTL-FICLONERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
> > +.SH NAME
> > +ioctl_ficlonerange, ioctl_ficlone \- share some the data of one file with another file
> > +.SH SYNOPSIS
> > +.br
> > +.B #include <sys/ioctl.h>
> > +.br
> > +.B #include <linux/fs.h>
> > +.sp
> > +.BI "int ioctl(int " dest_fd ", FICLONERANGE, struct file_clone_range * " arg );
> > +.br
> > +.BI "int ioctl(int " dest_fd ", FICLONE, int " src_fd );
> > +.SH DESCRIPTION
> > +If a filesystem supports files sharing physical storage between multiple
> > +files ("reflink"), this
> > +.BR ioctl (2)
> > +system call can be used to make some of the data in the
> > +.B src_fd
> > +file appear in the
> > +.B dest_fd
> > +file by sharing the underlying storage, which is faster than making a separate
> > +physical copy of the data.  If a file write should occur to a shared region,
> > +the filesystem must ensure that the changes remain private to the file being
> > +written.  This behavior is commonly referred to as "copy on write".
> > +
> > +This ioctl reflinks up to
> > +.IR src_length
> > +bytes from file descriptor
> > +.IR src_fd
> > +at offset
> > +.IR src_offset
> > +into the file
> > +.IR dest_fd
> > +at offset
> > +.IR dest_offset ",
> > +provided that both are files.  This information is conveyed in a structure of
> > +the following form:
> > +.in +4n
> > +.nf
> > +
> > +struct file_clone_range {
> > +        __s64 src_fd;
> > +        __u64 src_offset;
> > +        __u64 src_length;
> > +        __u64 dest_offset;
> > +};
> > +
> > +.fi
> > +.in
> > +Clones are atomic with regards to concurrent writes, so no locks need to be
> > +taken to obtain a consistent cloned copy.
> > +
> > +The FICLONE ioctl clones entire files.
> > +.SH RETURN VALUE
> > +On error, \-1 is returned, and
> > +.I errno
> > +is set to indicate the error.
> > +.PP
> > +.SH ERRORS
> > +Error codes can be one of, but are not limited to, the following:
> > +.TP
> > +.B EXDEV
> > +.IR dest_fd " and " src_fd
> > +are not on the same mounted filesystem.
> > +.TP
> > +.B EISDIR
> > +One of the files is a directory and the filesystem does not support shared
> > +regions in directories.
> > +.TP
> > +.B EINVAL
> > +The filesystem does not support reflinking the ranges of the given files.  This
> > +error can also appear if either file descriptor represents a device, fifo, or
> > +socket.  Disk filesystems generally require the offset and length arguments
> > +to be aligned to the fundamental block size.  XFS and btrfs do not support
> > +overlapping reflink ranges in the same file.
> > +.TP
> > +.B EBADF
> > +.IR src_fd
> > +is not open for reading;
> > +.IR dest_fd
> > +is not open for writing or is open for append-only writes; or the filesystem
> > +which
> > +.IR src_fd
> > +resides on does not support reflink.
> > +.TP
> > +.B EPERM
> > +.IR dest_fd
> > +is immutable.
> > +.TP
> > +.B ETXTBSY
> > +One of the files is a swap file.  Swap files cannot share storage.
> > +.TP
> > +.B EOPNOTSUPP
> > +This can appear if the filesystem does not support reflinking either file
> > +descriptor.
> > +.SH NOTES
> > +Because a copy on write operation requires the allocation of new storage, the
> > +.B fallocate (2)
> > +operation may un-share shared blocks to guarantee that subsequent writes will
> > +not fail because of lack of disk space.
> > +.SH CONFORMING TO
> > +This API is Linux-specific.  This ioctl was previously known as
> > +.B BTRFS_IOC_CLONE_RANGE
> > +and was private to btrfs.
> > +.fi
> > +.in
> > +.SH SEE ALSO
> > +.BR ioctl (2)
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-man" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] man2: document FICLONE and FICLONERANGE

[Michael Kerrisk (man-pages)]

On 03/15/2016 05:48 PM, Darrick J. Wong wrote:
> Document the FICLONE and FICLONERANGE ioctls, formerly known as the
> BTRFS_IOC_CLONE and BTRFS_IOC_CLONE_RANGE ioctls.

Thanks for the nice page, Darrick. Applied!

Cheers,

Michael



> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  man2/ioctl_ficlone.2      |    1 
>  man2/ioctl_ficlonerange.2 |  124 +++++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 125 insertions(+)
>  create mode 100644 man2/ioctl_ficlone.2
>  create mode 100644 man2/ioctl_ficlonerange.2
> 
> 
> diff --git a/man2/ioctl_ficlone.2 b/man2/ioctl_ficlone.2
> new file mode 100644
> index 0000000..19bb348
> --- /dev/null
> +++ b/man2/ioctl_ficlone.2
> @@ -0,0 +1 @@
> +.so man2/ioctl_ficlonerange.2
> diff --git a/man2/ioctl_ficlonerange.2 b/man2/ioctl_ficlonerange.2
> new file mode 100644
> index 0000000..c5c72aa
> --- /dev/null
> +++ b/man2/ioctl_ficlonerange.2
> @@ -0,0 +1,124 @@
> +.\" Copyright (C) 2016 Oracle.  All rights reserved.
> +.\"
> +.\" %%%LICENSE_START(VERBATIM)
> +.\" This program is free software; you can redistribute it and/or
> +.\" modify it under the terms of the GNU General Public License as
> +.\" published by the Free Software Foundation.
> +.\"
> +.\" This program is distributed in the hope that it would be useful,
> +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
> +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +.\" GNU General Public License for more details.
> +.\"
> +.\" You should have received a copy of the GNU General Public License
> +.\" along with this program; if not, write the Free Software Foundation,
> +.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +.\" %%%LICENSE_END
> +.TH IOCTL-FICLONERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
> +.SH NAME
> +ioctl_ficlonerange, ioctl_ficlone \- share some the data of one file with another file
> +.SH SYNOPSIS
> +.br
> +.B #include <sys/ioctl.h>
> +.br
> +.B #include <linux/fs.h>
> +.sp
> +.BI "int ioctl(int " dest_fd ", FICLONERANGE, struct file_clone_range * " arg );
> +.br
> +.BI "int ioctl(int " dest_fd ", FICLONE, int " src_fd );
> +.SH DESCRIPTION
> +If a filesystem supports files sharing physical storage between multiple
> +files ("reflink"), this
> +.BR ioctl (2)
> +system call can be used to make some of the data in the
> +.B src_fd
> +file appear in the
> +.B dest_fd
> +file by sharing the underlying storage, which is faster than making a separate
> +physical copy of the data.  If a file write should occur to a shared region,
> +the filesystem must ensure that the changes remain private to the file being
> +written.  This behavior is commonly referred to as "copy on write".
> +
> +This ioctl reflinks up to
> +.IR src_length
> +bytes from file descriptor
> +.IR src_fd
> +at offset
> +.IR src_offset
> +into the file
> +.IR dest_fd
> +at offset
> +.IR dest_offset ",
> +provided that both are files.  This information is conveyed in a structure of
> +the following form:
> +.in +4n
> +.nf
> +
> +struct file_clone_range {
> +        __s64 src_fd;
> +        __u64 src_offset;
> +        __u64 src_length;
> +        __u64 dest_offset;
> +};
> +
> +.fi
> +.in
> +Clones are atomic with regards to concurrent writes, so no locks need to be
> +taken to obtain a consistent cloned copy.
> +
> +The FICLONE ioctl clones entire files.
> +.SH RETURN VALUE
> +On error, \-1 is returned, and
> +.I errno
> +is set to indicate the error.
> +.PP
> +.SH ERRORS
> +Error codes can be one of, but are not limited to, the following:
> +.TP
> +.B EXDEV
> +.IR dest_fd " and " src_fd
> +are not on the same mounted filesystem.
> +.TP
> +.B EISDIR
> +One of the files is a directory and the filesystem does not support shared
> +regions in directories.
> +.TP
> +.B EINVAL
> +The filesystem does not support reflinking the ranges of the given files.  This
> +error can also appear if either file descriptor represents a device, fifo, or
> +socket.  Disk filesystems generally require the offset and length arguments
> +to be aligned to the fundamental block size.  XFS and btrfs do not support
> +overlapping reflink ranges in the same file.
> +.TP
> +.B EBADF
> +.IR src_fd
> +is not open for reading;
> +.IR dest_fd
> +is not open for writing or is open for append-only writes; or the filesystem
> +which
> +.IR src_fd
> +resides on does not support reflink.
> +.TP
> +.B EPERM
> +.IR dest_fd
> +is immutable.
> +.TP
> +.B ETXTBSY
> +One of the files is a swap file.  Swap files cannot share storage.
> +.TP
> +.B EOPNOTSUPP
> +This can appear if the filesystem does not support reflinking either file
> +descriptor.
> +.SH NOTES
> +Because a copy on write operation requires the allocation of new storage, the
> +.B fallocate (2)
> +operation may un-share shared blocks to guarantee that subsequent writes will
> +not fail because of lack of disk space.
> +.SH CONFORMING TO
> +This API is Linux-specific.  This ioctl was previously known as
> +.B BTRFS_IOC_CLONE_RANGE
> +and was private to btrfs.
> +.fi
> +.in
> +.SH SEE ALSO
> +.BR ioctl (2)
> 
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH 2/2] man2: document the FIDEDUPERANGE ioctl

[Michael Kerrisk (man-pages)]

On 03/15/2016 05:48 PM, Darrick J. Wong wrote:
> Document the FIDEDUPERANGE ioctl, formerly known as BTRFS_IOC_EXTENT_SAME.

Again, thanks for the nice page, Darrick. Applied!

Cheers,

Michael


> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
>  man2/ioctl_fideduperange.2 |  168 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 168 insertions(+)
>  create mode 100644 man2/ioctl_fideduperange.2
> 
> 
> diff --git a/man2/ioctl_fideduperange.2 b/man2/ioctl_fideduperange.2
> new file mode 100644
> index 0000000..0fbc388
> --- /dev/null
> +++ b/man2/ioctl_fideduperange.2
> @@ -0,0 +1,168 @@
> +.\" Copyright (C) 2016 Oracle.  All rights reserved.
> +.\"
> +.\" %%%LICENSE_START(VERBATIM)
> +.\" This program is free software; you can redistribute it and/or
> +.\" modify it under the terms of the GNU General Public License as
> +.\" published by the Free Software Foundation.
> +.\"
> +.\" This program is distributed in the hope that it would be useful,
> +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
> +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +.\" GNU General Public License for more details.
> +.\"
> +.\" You should have received a copy of the GNU General Public License
> +.\" along with this program; if not, write the Free Software Foundation,
> +.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +.\" %%%LICENSE_END
> +.TH IOCTL-FIDEDUPERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
> +.SH NAME
> +ioctl_fideduperange \- share some the data of one file with another file
> +.SH SYNOPSIS
> +.br
> +.B #include <sys/ioctl.h>
> +.br
> +.B #include <linux/fs.h>
> +.sp
> +.BI "int ioctl(int " src_fd ", FIDEDUPERANGE, struct file_dedupe_range * " arg );
> +.SH DESCRIPTION
> +If a filesystem supports files sharing physical storage between multiple
> +files, this
> +.BR ioctl (2)
> +system call can be used to make some of the data in the
> +.B src_fd
> +file appear in the
> +.B dest_fd
> +file by sharing the underlying storage if the file data is identical
> +("deduplication").  This reduces storage consumption by allowing the filesystem
> +to store one shared copy of the data.  If a file write should occur to a shared
> +region, the filesystem must ensure that the changes remain private to the file
> +being written.  This behavior is commonly referred to as "copy on write".
> +
> +This ioctl performs the "compare and share if identical" operation on up to
> +.IR src_length
> +bytes from file descriptor
> +.IR src_fd
> +at offset
> +.IR src_offset ".
> +This information is conveyed in a structure of the following form:
> +.in +4n
> +.nf
> +
> +struct file_dedupe_range {
> +        __u64 src_offset;
> +        __u64 src_length;
> +        __u16 dest_count;
> +        __u16 reserved1;
> +        __u32 reserved2;
> +        struct file_dedupe_range_info info[0];
> +};
> +.fi
> +.in
> +Deduplication is atomic with regards to concurrent writes, so no locks need to
> +be taken to obtain a consistent deduplicated copy.
> +
> +The fields 
> +.IR reserved1 " and " reserved2
> +must be zero.
> +
> +Destinations for the deduplication operation are conveyed in the array at the
> +end of the structure.  The number of destinations is given in
> +.IR dest_count ",
> +and the destination information is conveyed in the following form:
> +
> +.in +4n
> +.nf
> +struct file_dedupe_range_info {
> +        __s64 dest_fd;
> +        __u64 dest_offset;
> +        __u64 bytes_deduped;
> +        __s32 status;
> +        __u32 reserved;
> +};
> +
> +.fi
> +.in
> +
> +Each deduplication operation targets
> +.IR length
> +bytes in file descriptor
> +.IR dest_fd
> +at offset
> +.IR logical_offset ".
> +The field
> +.IR reserved
> +must be zero.
> +
> +Upon successful completion of this ioctl, the number of bytes successfully
> +deduplicated is returned in
> +.IR bytes_deduped
> +and a status code for the deduplication operation is returned in
> +.IR status ".
> +
> +The
> +.IR status
> +code is set to
> +.B 0
> +for success, a negative error code in case of error, or
> +.B FILE_DEDUPE_RANGE_DIFFERS
> +if the data did not match.
> +
> +.SH RETURN VALUE
> +On error, \-1 is returned, and
> +.I errno
> +is set to indicate the error.
> +.PP
> +.SH ERRORS
> +Error codes can be one of, but are not limited to, the following:
> +.TP
> +.B EXDEV
> +.IR dest_fd " and " src_fd
> +are not on the same mounted filesystem.
> +.TP
> +.B EISDIR
> +One of the files is a directory and the filesystem does not support shared
> +regions in directories.
> +.TP
> +.B EINVAL
> +The filesystem does not support deduplicating the ranges of the given files.
> +This error can also appear if either file descriptor represents a device, fifo,
> +or socket.  Disk filesystems generally require the offset and length arguments
> +to be aligned to the fundamental block size.  Neither btrfs nor XFS support
> +overlapping deduplication ranges in the same file.
> +.TP
> +.B EBADF
> +.IR src_fd
> +is not open for reading;
> +.IR dest_fd
> +is not open for writing or is open for append-only writes; or the filesystem
> +which
> +.IR src_fd
> +resides on does not support deduplication.
> +.TP
> +.B EPERM
> +.IR dest_fd
> +is immutable.
> +.TP
> +.B ETXTBSY
> +One of the files is a swap file.  Swap files cannot share storage.
> +.TP
> +.B EOPNOTSUPP
> +This can appear if the filesystem does not support deduplicating either file
> +descriptor.
> +.SH NOTES
> +Because a copy on write operation requires the allocation of new storage, the
> +.B fallocate (2)
> +operation may un-share shared blocks to guarantee that subsequent writes will
> +not fail because of lack of disk space.
> +
> +Some filesystems may limit the amount of data that can be deduplicated in a
> +single call.
> +
> +.SH CONFORMING TO
> +This API is Linux-specific.  This ioctl was previously known as
> +.B BTRFS_IOC_FILE_EXTENT_SAME
> +and was private to btrfs.
> +.fi
> +.in
> +.SH SEE ALSO
> +.BR ioctl (2)
> 
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH 2/2] man2: document the FIDEDUPERANGE ioctl

[Darrick J. Wong]

On Wed, Jun 08, 2016 at 12:35:47PM +0200, Michael Kerrisk (man-pages) wrote:
> On 03/15/2016 05:48 PM, Darrick J. Wong wrote:
> > Document the FIDEDUPERANGE ioctl, formerly known as BTRFS_IOC_EXTENT_SAME.
> 
> Again, thanks for the nice page, Darrick. Applied!

NP.  I have a couple of one-line fixes for the manpages that Christoph
suggested a while back that I'll send separately.

[Sorry I forgot about the new-sentence->new-line rule.]

--D

> 
> Cheers,
> 
> Michael
> 
> 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> >  man2/ioctl_fideduperange.2 |  168 ++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 168 insertions(+)
> >  create mode 100644 man2/ioctl_fideduperange.2
> > 
> > 
> > diff --git a/man2/ioctl_fideduperange.2 b/man2/ioctl_fideduperange.2
> > new file mode 100644
> > index 0000000..0fbc388
> > --- /dev/null
> > +++ b/man2/ioctl_fideduperange.2
> > @@ -0,0 +1,168 @@
> > +.\" Copyright (C) 2016 Oracle.  All rights reserved.
> > +.\"
> > +.\" %%%LICENSE_START(VERBATIM)
> > +.\" This program is free software; you can redistribute it and/or
> > +.\" modify it under the terms of the GNU General Public License as
> > +.\" published by the Free Software Foundation.
> > +.\"
> > +.\" This program is distributed in the hope that it would be useful,
> > +.\" but WITHOUT ANY WARRANTY; without even the implied warranty of
> > +.\" MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > +.\" GNU General Public License for more details.
> > +.\"
> > +.\" You should have received a copy of the GNU General Public License
> > +.\" along with this program; if not, write the Free Software Foundation,
> > +.\" Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> > +.\" %%%LICENSE_END
> > +.TH IOCTL-FIDEDUPERANGE 2 2016-02-10 "Linux" "Linux Programmer's Manual"
> > +.SH NAME
> > +ioctl_fideduperange \- share some the data of one file with another file
> > +.SH SYNOPSIS
> > +.br
> > +.B #include <sys/ioctl.h>
> > +.br
> > +.B #include <linux/fs.h>
> > +.sp
> > +.BI "int ioctl(int " src_fd ", FIDEDUPERANGE, struct file_dedupe_range * " arg );
> > +.SH DESCRIPTION
> > +If a filesystem supports files sharing physical storage between multiple
> > +files, this
> > +.BR ioctl (2)
> > +system call can be used to make some of the data in the
> > +.B src_fd
> > +file appear in the
> > +.B dest_fd
> > +file by sharing the underlying storage if the file data is identical
> > +("deduplication").  This reduces storage consumption by allowing the filesystem
> > +to store one shared copy of the data.  If a file write should occur to a shared
> > +region, the filesystem must ensure that the changes remain private to the file
> > +being written.  This behavior is commonly referred to as "copy on write".
> > +
> > +This ioctl performs the "compare and share if identical" operation on up to
> > +.IR src_length
> > +bytes from file descriptor
> > +.IR src_fd
> > +at offset
> > +.IR src_offset ".
> > +This information is conveyed in a structure of the following form:
> > +.in +4n
> > +.nf
> > +
> > +struct file_dedupe_range {
> > +        __u64 src_offset;
> > +        __u64 src_length;
> > +        __u16 dest_count;
> > +        __u16 reserved1;
> > +        __u32 reserved2;
> > +        struct file_dedupe_range_info info[0];
> > +};
> > +.fi
> > +.in
> > +Deduplication is atomic with regards to concurrent writes, so no locks need to
> > +be taken to obtain a consistent deduplicated copy.
> > +
> > +The fields 
> > +.IR reserved1 " and " reserved2
> > +must be zero.
> > +
> > +Destinations for the deduplication operation are conveyed in the array at the
> > +end of the structure.  The number of destinations is given in
> > +.IR dest_count ",
> > +and the destination information is conveyed in the following form:
> > +
> > +.in +4n
> > +.nf
> > +struct file_dedupe_range_info {
> > +        __s64 dest_fd;
> > +        __u64 dest_offset;
> > +        __u64 bytes_deduped;
> > +        __s32 status;
> > +        __u32 reserved;
> > +};
> > +
> > +.fi
> > +.in
> > +
> > +Each deduplication operation targets
> > +.IR length
> > +bytes in file descriptor
> > +.IR dest_fd
> > +at offset
> > +.IR logical_offset ".
> > +The field
> > +.IR reserved
> > +must be zero.
> > +
> > +Upon successful completion of this ioctl, the number of bytes successfully
> > +deduplicated is returned in
> > +.IR bytes_deduped
> > +and a status code for the deduplication operation is returned in
> > +.IR status ".
> > +
> > +The
> > +.IR status
> > +code is set to
> > +.B 0
> > +for success, a negative error code in case of error, or
> > +.B FILE_DEDUPE_RANGE_DIFFERS
> > +if the data did not match.
> > +
> > +.SH RETURN VALUE
> > +On error, \-1 is returned, and
> > +.I errno
> > +is set to indicate the error.
> > +.PP
> > +.SH ERRORS
> > +Error codes can be one of, but are not limited to, the following:
> > +.TP
> > +.B EXDEV
> > +.IR dest_fd " and " src_fd
> > +are not on the same mounted filesystem.
> > +.TP
> > +.B EISDIR
> > +One of the files is a directory and the filesystem does not support shared
> > +regions in directories.
> > +.TP
> > +.B EINVAL
> > +The filesystem does not support deduplicating the ranges of the given files.
> > +This error can also appear if either file descriptor represents a device, fifo,
> > +or socket.  Disk filesystems generally require the offset and length arguments
> > +to be aligned to the fundamental block size.  Neither btrfs nor XFS support
> > +overlapping deduplication ranges in the same file.
> > +.TP
> > +.B EBADF
> > +.IR src_fd
> > +is not open for reading;
> > +.IR dest_fd
> > +is not open for writing or is open for append-only writes; or the filesystem
> > +which
> > +.IR src_fd
> > +resides on does not support deduplication.
> > +.TP
> > +.B EPERM
> > +.IR dest_fd
> > +is immutable.
> > +.TP
> > +.B ETXTBSY
> > +One of the files is a swap file.  Swap files cannot share storage.
> > +.TP
> > +.B EOPNOTSUPP
> > +This can appear if the filesystem does not support deduplicating either file
> > +descriptor.
> > +.SH NOTES
> > +Because a copy on write operation requires the allocation of new storage, the
> > +.B fallocate (2)
> > +operation may un-share shared blocks to guarantee that subsequent writes will
> > +not fail because of lack of disk space.
> > +
> > +Some filesystems may limit the amount of data that can be deduplicated in a
> > +single call.
> > +
> > +.SH CONFORMING TO
> > +This API is Linux-specific.  This ioctl was previously known as
> > +.B BTRFS_IOC_FILE_EXTENT_SAME
> > +and was private to btrfs.
> > +.fi
> > +.in
> > +.SH SEE ALSO
> > +.BR ioctl (2)
> > 
> > 
> 
> 
> -- 
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-api" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html