linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Steve French <smfrench@gmail.com>
To: Amir Goldstein <amir73il@gmail.com>
Cc: "Alejandro Colomar (man-pages)" <alx.manpages@gmail.com>,
	Luis Henriques <lhenriques@suse.de>,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	Anna Schumaker <anna.schumaker@netapp.com>,
	Jeff Layton <jlayton@kernel.org>,
	Steve French <sfrench@samba.org>,
	Miklos Szeredi <miklos@szeredi.hu>,
	Trond Myklebust <trond.myklebust@hammerspace.com>,
	Alexander Viro <viro@zeniv.linux.org.uk>,
	"Darrick J. Wong" <darrick.wong@oracle.com>,
	Dave Chinner <dchinner@redhat.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	Nicolas Boichat <drinkcat@chromium.org>,
	Ian Lance Taylor <iant@google.com>,
	Luis Lozano <llozano@chromium.org>,
	Andreas Dilger <adilger@dilger.ca>,
	Olga Kornievskaia <aglo@umich.edu>,
	Christoph Hellwig <hch@infradead.org>,
	ceph-devel <ceph-devel@vger.kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	CIFS <linux-cifs@vger.kernel.org>,
	samba-technical <samba-technical@lists.samba.org>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
	linux-man <linux-man@vger.kernel.org>
Subject: Re: [PATCH] copy_file_range.2: Kernel v5.12 updates
Date: Sat, 27 Feb 2021 17:08:11 -0600	[thread overview]
Message-ID: <CAH2r5mvJPh6H_Owt7QiBY0xWO0T6ai65R5tspn+cDru0_P0V4A@mail.gmail.com> (raw)
In-Reply-To: <CAOQ4uxgFCBNwRD7e1srwaVrZMGfOE_JXENL4Q2En52srdj2AYA@mail.gmail.com>

On Fri, Feb 26, 2021 at 11:43 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Sat, Feb 27, 2021 at 12:19 AM Alejandro Colomar (man-pages)
> <alx.manpages@gmail.com> wrote:
> >
> > Hello Amir, Luis,
> >
> > On 2/24/21 5:10 PM, Amir Goldstein wrote:
> > > On Wed, Feb 24, 2021 at 4:22 PM Luis Henriques <lhenriques@suse.de> wrote:
> > >>
> > >> Update man-page with recent changes to this syscall.
> > >>
> > >> Signed-off-by: Luis Henriques <lhenriques@suse.de>
> > >> ---
> > >> Hi!
> > >>
> > >> Here's a suggestion for fixing the manpage for copy_file_range().  Note that
> > >> I've assumed the fix will hit 5.12.
> > >>
> > >>   man2/copy_file_range.2 | 10 +++++++++-
> > >>   1 file changed, 9 insertions(+), 1 deletion(-)
> > >>
> > >> diff --git a/man2/copy_file_range.2 b/man2/copy_file_range.2
> > >> index 611a39b8026b..b0fd85e2631e 100644
> > >> --- a/man2/copy_file_range.2
> > >> +++ b/man2/copy_file_range.2
> > >> @@ -169,6 +169,9 @@ Out of memory.
> > >>   .B ENOSPC
> > >>   There is not enough space on the target filesystem to complete the copy.
> > >>   .TP
> > >> +.B EOPNOTSUPP
> >
> > I'll add the kernel version here:
> >
> > .BR EOPNOTSUPP " (since Linux 5.12)"
>
> Error could be returned prior to 5.3 and would be probably returned
> by future stable kernels 5.3..5.12 too
>
> >
> > >> +The filesystem does not support this operation >> +.TP
> > >>   .B EOVERFLOW
> > >>   The requested source or destination range is too large to represent in the
> > >>   specified data types.
> > >> @@ -187,7 +190,7 @@ refers to an active swap file.
> > >>   .B EXDEV
> > >>   The files referred to by
> > >>   .IR fd_in " and " fd_out
> > >> -are not on the same mounted filesystem (pre Linux 5.3).
> > >> +are not on the same mounted filesystem (pre Linux 5.3 and post Linux 5.12).
> >
> > I'm not sure that 'mounted' adds any value here.  Would you remove the
> > word here?
>
> See rename(2). 'mounted' in this context is explained there.
> HOWEVER, it does not fit here.
> copy_file_range() IS allowed between two mounts of the same filesystem instance.
>
> To make things more complicated, it appears that cross mount clone is not
> allowed via FICLONE/FICLONERANGE ioctl, so ioctl_ficlonerange(2) man page
> also uses the 'mounted filesystem' terminology for EXDEV
>
> As things stand now, because of the fallback to clone logic,
> copy_file_range() provides a way for users to clone across different mounts
> of the same filesystem instance, which they cannot do with the FICLONE ioctl.
>
> Fun :)
>
> BTW, I don't know if preventing cross mount clone was done intentionally,
> but as I wrote in a comment in the code once:
>
>         /*
>          * FICLONE/FICLONERANGE ioctls enforce that src and dest files are on
>          * the same mount. Practically, they only need to be on the same file
>          * system.
>          */
>
> >
> > It reads as if two separate devices with the same filesystem type would
> > still give this error.
> >
> > Per the LWN.net article Amir shared, this is permitted ("When called
> > from user space, copy_file_range() will only try to copy a file across
> > filesystems if the two are of the same type").
> >
> > This behavior was slightly different before 5.3 AFAICR (was it?) ("until
> > then, copy_file_range() refused to copy between files that were not
> > located on the same filesystem.").  If that's the case, I'd specify the
> > difference, or more probably split the error into two, one before 5.3,
> > and one since 5.12.
> >
>
> True.
>
> > >
> > > I think you need to drop the (Linux range) altogether.
> >
> > I'll keep the range.  Users of 5.3..5.11 might be surprised if the
> > filesystems are different and they don't get an error, I think.
> >
> > I reworded it to follow other pages conventions:
> >
> > .BR EXDEV " (before Linux 5.3; or since Linux 5.12)"
> >
> > which renders as:
> >
> >         EXDEV (before Linux 5.3; or since Linux 5.12)
> >                The files referred to by fd_in and fd_out are not on
> >                the same mounted filesystem.
> >
>
> drop 'mounted'
>
> >
> > > What's missing here is the NFS cross server copy use case.
> > > Maybe:

At least for the SMB3 kernel server (ksmbd "cifsd") looks like they use splice.
And for the user space CIFS/SMB3 server (like Samba) they have a configurable
plug in library interface ("Samba VFS modules") that would allow you
to implement
cross filesystem copy optimally for your version of Linux and plug
this into Samba
with little work on your part.

> >
> > Again, this wasn't true before 5.3, right?
> >
>
> Right.
> Actually, v5.3 provides the vfs capabilities for filesystems to support
> cross fs copy. I am not sure if NFS already implements cross fs copy in
> v5.3 and not sure about cifs. Need to get input from nfs/cis developers
> or dig in the release notes for server-side copy.

The SMB3 protocol has multiple ways to do "server side copy" (copy
offload to the server), some of which would apply to your example.
The case of "reflink" in many cases would be most efficient, and is supported
by the Linux client (see MS-SMB2 protocol specification section 3.3.5.15.18) but
is supported by fewer server file systems, so probably more important
to focus on
the other mechanisms which are server side copy rather than clone.  The most
popular way, supported by most servers, is  "CopyChunk" - 100s of
millions of systems
support this (if not more) - see MS-SMB2 protocol specification
section 2.2.31.1 and
3.3.5.15.16 - there are various cases where two different SMB3 mounts
on the same
client could handle cross mount server side copy.

There are other mechanisms supported by fewer servers SMB3 ODX/T10 style copy
offload (Windows and some others see e.g. Gordon at Nexenta's presentation
https://www.slideshare.net/gordonross/smb3-offload-data-transfer-odx)
but still popular for virtualization workloads.  For this it could be
even more common
for those to be different mounts on the client.  The Linux client does
not support
the SMB3 ODX/T10 offload yet but it would be good to add support for it.
There is a nice description of its additional benefits at
https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/offloaded-data-transfer

But - yes SMB3 on Linux can have cross mount file copy today, which is
far more efficient
(having the server do the copy for us) rather than sending large
reads/writes back and
forth over the network from the client.  In the future I am hoping that use case
becomes even more common over SMB3 as cloud servers improve.


> > > You may refer the reader to VERSIONS section where it will say which
> > > filesystems support cross-fs copy as of kernel version XXX (i.e. cifs and nfs).
> > >
> > >>   .SH VERSIONS
> > >>   The
> > >>   .BR copy_file_range ()
> > >> @@ -202,6 +205,11 @@ Applications should target the behaviour and requirements of 5.3 kernels.
> > >>   .PP
> > >>   First support for cross-filesystem copies was introduced in Linux 5.3.
> > >>   Older kernels will return -EXDEV when cross-filesystem copies are attempted.
> > >> +.PP
> > >> +After Linux 5.12, support for copies between different filesystems was dropped.
> > >> +However, individual filesystems may still provide
> > >> +.BR copy_file_range ()
> > >> +implementations that allow copies across different devices.

Yes - this could be very important, especially for cifs (smb3) going forward.



-- 
Thanks,

Steve

  parent reply	other threads:[~2021-02-27 23:09 UTC|newest]

Thread overview: 136+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-02-12  4:43 [PATCH 0/6] Add generated flag to filesystem struct to block copy_file_range Nicolas Boichat
2021-02-12  4:44 ` [PATCH 1/6] fs: Add flag to file_system_type to indicate content is generated Nicolas Boichat
2021-02-12  7:46   ` Greg KH
2021-02-12  8:20     ` Nicolas Boichat
2021-02-12  8:37       ` Greg KH
2021-02-12 15:33         ` Ian Lance Taylor
2021-02-12 15:45           ` Greg KH
2021-02-12 15:59             ` Ian Lance Taylor
2021-02-12 16:28               ` Greg KH
2021-02-12 20:22                 ` Ian Lance Taylor
2021-02-12 23:03             ` Dave Chinner
2021-02-12 23:07               ` Ian Lance Taylor
2021-02-12 23:27                 ` Dave Chinner
2021-02-12 23:54                   ` Darrick J. Wong
2021-02-15  0:38                     ` Dave Chinner
2021-02-15  1:12                       ` Ian Lance Taylor
2021-02-15  1:25                         ` Nicolas Boichat
2021-02-15  5:56                           ` Amir Goldstein
2021-02-15  8:30                           ` Greg KH
2021-02-12  8:22     ` Amir Goldstein
2021-02-12  8:39       ` Greg KH
2021-02-12 12:05         ` Luis Henriques
2021-02-12 12:18           ` Greg KH
2021-02-12 12:41             ` Luis Henriques
2021-02-12 14:11               ` Greg KH
2021-02-12 15:01                 ` Luis Henriques
2021-02-15  6:12               ` Amir Goldstein
2021-02-15 10:39                 ` Luis Henriques
2021-02-15 12:22                   ` Luis Henriques
2021-02-15 14:23                     ` Amir Goldstein
2021-02-15 14:51                       ` Luis Henriques
2021-02-15 15:43                       ` [PATCH v2] vfs: prevent copy_file_range to copy across devices Luis Henriques
2021-02-15 16:02                         ` Trond Myklebust
2021-02-16  0:25                           ` Steve French
2021-02-15 16:34                         ` Amir Goldstein
2021-02-15 16:53                           ` Trond Myklebust
2021-02-15 17:24                             ` Amir Goldstein
2021-02-15 18:57                               ` Trond Myklebust
2021-02-15 19:43                                 ` Amir Goldstein
2021-02-16 11:17                                   ` Luis Henriques
2021-02-16 11:28                                     ` gregkh
2021-02-16 12:01                                       ` Luis Henriques
2021-02-16 12:08                                         ` Greg KH
2021-02-16 13:51                                     ` Amir Goldstein
2021-02-16 16:42                                       ` Luis Henriques
2021-02-16 17:44                                         ` Amir Goldstein
2021-02-16 18:55                                           ` Luis Henriques
2021-02-16 19:20                                             ` Amir Goldstein
2021-02-16 19:27                                               ` Anna Schumaker
2021-02-16 19:31                                                 ` Steve French
2021-02-16 19:40                                                   ` Amir Goldstein
2021-02-16 21:15                                                     ` Steve French
2021-02-17  8:08                                                       ` Amir Goldstein
2021-02-17 17:26                                                         ` [PATCH v3] vfs: fix copy_file_range regression in cross-fs copies Luis Henriques
2021-02-17 20:47                                                           ` Amir Goldstein
2021-02-18  0:56                                                           ` Nicolas Boichat
2021-02-18  5:32                                                           ` Olga Kornievskaia
2021-02-18  6:47                                                             ` Amir Goldstein
2021-02-18 16:28                                                               ` Olga Kornievskaia
2021-02-18  7:43                                                           ` Christoph Hellwig
2021-02-18  0:50                                                         ` [PATCH v2] vfs: prevent copy_file_range to copy across devices Andreas Dilger
2021-02-18  7:34                                                           ` gregkh
2021-02-16 18:54                                       ` Andreas Dilger
2021-02-17  4:45                         ` Nicolas Boichat
2021-02-18  7:42                         ` Christoph Hellwig
2021-02-18  9:10                           ` Amir Goldstein
2021-02-18 10:29                             ` Luis Henriques
2021-02-18 12:15                               ` Luis Henriques
2021-02-18 12:49                                 ` Amir Goldstein
2021-02-18 14:36                                   ` [PATCH v4] vfs: fix copy_file_range regression in cross-fs copies Luis Henriques
2021-02-18 14:58                                     ` Amir Goldstein
2021-02-18 15:17                                       ` [PATCH v5] " Luis Henriques
2021-02-18 15:53                                         ` Amir Goldstein
2021-02-18 16:35                                           ` Luis Henriques
2021-02-18 17:18                                             ` [PATCH v6] " Luis Henriques
2021-02-19 21:18                                               ` Olga Kornievskaia
2021-02-19 21:52                                                 ` Amir Goldstein
2021-02-21 19:58                                                 ` [PATCH v7] " Luis Henriques
2021-02-22  3:00                                                   ` Nicolas Boichat
2021-02-22 10:24                                                   ` [PATCH v8] " Luis Henriques
2021-02-22 10:46                                                     ` Amir Goldstein
2021-02-22 16:25                                                     ` dai.ngo
2021-02-23 10:32                                                       ` Luis Henriques
2021-02-23 15:28                                                         ` Amir Goldstein
2021-02-23 15:29                                                         ` dai.ngo
2021-02-23 16:02                                                           ` dai.ngo
2021-02-23 16:47                                                             ` Amir Goldstein
2021-02-23 16:57                                                               ` dai.ngo
     [not found]                                                                 ` <e3eed18b-fc7e-e687-608b-7f662017329c@oracle.com>
2021-02-23 17:33                                                                   ` Amir Goldstein
2021-02-24  0:13                                                                     ` dai.ngo
2021-02-23 17:56                                                                 ` Luis Henriques
2021-02-23 17:13                                                             ` Olga Kornievskaia
2021-02-24  1:00                                                     ` Olga Kornievskaia
2021-02-24 10:23                                                       ` Luis Henriques
2021-02-24 10:44                                                         ` Nicolas Boichat
2021-04-09  5:23                                                           ` Nicolas Boichat
2021-04-09 13:39                                                             ` Luis Henriques
2021-04-09 13:50                                                               ` Amir Goldstein
2021-04-23  4:40                                                                 ` Nicolas Boichat
2021-05-03  8:54                                                                   ` Luis Henriques
2021-05-10  4:59                                                                     ` Amir Goldstein
2021-05-10  9:10                                                                       ` Luis Henriques
2021-02-24 14:23                                                     ` [PATCH] copy_file_range.2: Kernel v5.12 updates Luis Henriques
2021-02-24 16:10                                                       ` Amir Goldstein
2021-02-25 10:21                                                         ` Luis Henriques
2021-02-26 10:13                                                           ` Alejandro Colomar (man-pages)
2021-02-26 10:34                                                             ` Amir Goldstein
2021-02-26 11:15                                                               ` Alejandro Colomar (man-pages)
2021-02-26 13:59                                                                 ` Jeff Layton
2021-02-26 21:26                                                                   ` Alejandro Colomar (man-pages)
2021-02-26 22:18                                                         ` Alejandro Colomar (man-pages)
2021-02-27  5:41                                                           ` Amir Goldstein
2021-02-27 12:20                                                             ` Alejandro Colomar (man-pages)
2021-02-27 13:49                                                               ` [RFC v2] copy_file_range.2: Update cross-filesystem support for 5.12 Alejandro Colomar
2021-02-27 16:00                                                                 ` Amir Goldstein
2021-02-27 23:08                                                             ` Steve French [this message]
2021-02-28  7:35                                                               ` [PATCH] copy_file_range.2: Kernel v5.12 updates Amir Goldstein
2021-02-28 22:25                                                                 ` Steve French
2021-03-01  6:18                                                                   ` Amir Goldstein
2021-03-01 14:41                                                       ` [RFC v3] copy_file_range.2: Update cross-filesystem support for 5.12 Alejandro Colomar
2021-03-01 14:58                                                         ` Amir Goldstein
2021-03-04  9:38                                                       ` [RFC v4] " Alejandro Colomar
2021-03-04 17:13                                                         ` Darrick J. Wong
2021-03-04 18:24                                                           ` Alejandro Colomar (man-pages)
2021-03-04 23:50                                                             ` Darrick J. Wong
2021-02-24  7:15                                     ` [PATCH v4] vfs: fix copy_file_range regression in cross-fs copies Amir Goldstein
2021-02-24  8:30                                       ` Petr Vorel
2021-02-18 20:41                             ` [PATCH v2] vfs: prevent copy_file_range to copy across devices Steve French
2021-02-12 23:15       ` [PATCH 1/6] fs: Add flag to file_system_type to indicate content is generated Dave Chinner
2021-02-12  7:54   ` Amir Goldstein
2021-02-12  4:44 ` [PATCH 2/6] proc: Add FS_GENERATED_CONTENT to filesystem flags Nicolas Boichat
2021-02-12  4:44 ` [PATCH 6/6] vfs: Disallow copy_file_range on generated file systems Nicolas Boichat
2021-02-12  4:53   ` Darrick J. Wong
2021-02-12  4:59     ` Darrick J. Wong
2021-02-12  5:24       ` Nicolas Boichat
2021-02-14 23:09 ` [PATCH 0/6] Add generated flag to filesystem struct to block copy_file_range Al Viro

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAH2r5mvJPh6H_Owt7QiBY0xWO0T6ai65R5tspn+cDru0_P0V4A@mail.gmail.com \
    --to=smfrench@gmail.com \
    --cc=adilger@dilger.ca \
    --cc=aglo@umich.edu \
    --cc=alx.manpages@gmail.com \
    --cc=amir73il@gmail.com \
    --cc=anna.schumaker@netapp.com \
    --cc=ceph-devel@vger.kernel.org \
    --cc=darrick.wong@oracle.com \
    --cc=dchinner@redhat.com \
    --cc=drinkcat@chromium.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hch@infradead.org \
    --cc=iant@google.com \
    --cc=jlayton@kernel.org \
    --cc=lhenriques@suse.de \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-man@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=llozano@chromium.org \
    --cc=miklos@szeredi.hu \
    --cc=mtk.manpages@gmail.com \
    --cc=samba-technical@lists.samba.org \
    --cc=sfrench@samba.org \
    --cc=trond.myklebust@hammerspace.com \
    --cc=viro@zeniv.linux.org.uk \
    --subject='Re: [PATCH] copy_file_range.2: Kernel v5.12 updates' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).