cp --reflink invalid argument error

cp --reflink invalid argument error

[Brian Hansen]

Hello,

First time i've sent to this group but I am trying to figure out the
cause of this. Normal copy is working fine, but then if I use
--reflink it says invalid argument. Not sure how to read some of this,
but here is the strace.

I'm running kernel v4.15

Here is the full output of strace. I ran a strace on normal copy and
most looks similar so I'm not able to figure out much here...

https://pastebin.com/raw/YmQ8FvCH

reflink copy now works with nocow?

[Roman Mamedov]

On Sat, 2 Nov 2019 08:49:37 -0500
Brian Hansen <dulanic@gmail.com> wrote:

> Hello,
> 
> First time i've sent to this group but I am trying to figure out the
> cause of this. Normal copy is working fine, but then if I use
> --reflink it says invalid argument. Not sure how to read some of this,
> but here is the strace.
> 
> I'm running kernel v4.15
> 
> Here is the full output of strace. I ran a strace on normal copy and
> most looks similar so I'm not able to figure out much here...
> 
> https://pastebin.com/raw/YmQ8FvCH

At first I was going to say, "oh it's because you are using 'chattr +C', or
mounted the filesystem as nocow, and reflink copying is prevented by those".
In fact this article from 2014 confirms that to be the case:
http://infotinks.com/btrfs-nodatacow-reflink-copies-snapshots/

But then I tested on my machine, and what used to fail, now works:

  # mkdir tmp
  # chattr +C tmp
  # echo abc > tmp/a
  # cp -a --reflink=always tmp/a tmp/b
  # lsattr tmp/
  ----------------C-- tmp/a
  ----------------C-- tmp/b

According to strace, the clone IOCTL succeeds:

...
  openat(AT_FDCWD, "tmp/b", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
  fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
  ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = 0
...

Same on kernels 4.14.151, 4.14.113 and 4.9.189.

So I wonder, is setting nocow via 'chattr +C' getting ignored now, or is there
an improvement that it no longer prevents reflink copying?

-- 
With respect,
Roman

Re: reflink copy now works with nocow?

[Brian Hansen]

Ah I see, so I only have +C set on my temporary directory, but I guess
when the file is moved to the final directory which does not have C
set, it still has it enabled from when it was in the temporary
directory. So I'll need to setup a script to remove C when moving
directories. Unless there is a easier way to deal with this? I have it
download to incomplete then move to a final dir, but it seems to trake
the +C /w it.

------------------ ./archive
------------------ ./radarr
------------------ ./sonarr
------------------ ./autodl
------------------ ./radarr4k
---------------C-- ./incomplete
------------------ ./archive1
------------------ ./temp
------------------ ./freeleech

File was moved from incomplete to radarr4k but it seems to carry the +C.

---------------C--
/mnt/btrfs/downloads/torrent/radarr4k/Spider-Man.Far.from.Home.2019.UHD.BluRay.2160p.TrueHD.Atmos.7.1.HEVC.REMUX-FraMeSToR/Spider-Man.Far.from.Home.2019.UHD.BluRay.2160p.TrueHD.Atmos.7.1.HEVC.REMUX-FraMeSToR.mkv


On Sat, Nov 2, 2019 at 9:36 AM Roman Mamedov <rm@romanrm.net> wrote:
>
> On Sat, 2 Nov 2019 08:49:37 -0500
> Brian Hansen <dulanic@gmail.com> wrote:
>
> > Hello,
> >
> > First time i've sent to this group but I am trying to figure out the
> > cause of this. Normal copy is working fine, but then if I use
> > --reflink it says invalid argument. Not sure how to read some of this,
> > but here is the strace.
> >
> > I'm running kernel v4.15
> >
> > Here is the full output of strace. I ran a strace on normal copy and
> > most looks similar so I'm not able to figure out much here...
> >
> > https://pastebin.com/raw/YmQ8FvCH
>
> At first I was going to say, "oh it's because you are using 'chattr +C', or
> mounted the filesystem as nocow, and reflink copying is prevented by those".
> In fact this article from 2014 confirms that to be the case:
> http://infotinks.com/btrfs-nodatacow-reflink-copies-snapshots/
>
> But then I tested on my machine, and what used to fail, now works:
>
>   # mkdir tmp
>   # chattr +C tmp
>   # echo abc > tmp/a
>   # cp -a --reflink=always tmp/a tmp/b
>   # lsattr tmp/
>   ----------------C-- tmp/a
>   ----------------C-- tmp/b
>
> According to strace, the clone IOCTL succeeds:
>
> ...
>   openat(AT_FDCWD, "tmp/b", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
>   fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
>   ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = 0
> ...
>
> Same on kernels 4.14.151, 4.14.113 and 4.9.189.
>
> So I wonder, is setting nocow via 'chattr +C' getting ignored now, or is there
> an improvement that it no longer prevents reflink copying?
>
> --
> With respect,
> Roman

Re: reflink copy now works with nocow?

[Roman Mamedov]

On Sat, 2 Nov 2019 10:09:27 -0500
Brian Hansen <dulanic@gmail.com> wrote:

> So I'll need to setup a script to remove C when moving
> directories.

The only way to un-nocow a +C file is to physically rewrite it into a new
copy. The +C attribute can be removed on existing files only if they are 0
bytes in size.

> Unless there is a easier way to deal with this?

Try a different/newer kernel version, as noted in my message all of this
doesn't seem to matter anymore, and nocow files can now be reflinked (or +C is
not working anymore and they aren't actually nocow).

-- 
With respect,
Roman

Re: reflink copy now works with nocow?

[Brian Hansen]

I updated kernel to 5.0.0-32-generic and the same issue occurs. Not sure.

https://pastebin.com/LikPgNMt

On Sat, Nov 2, 2019 at 10:24 AM Roman Mamedov <rm@romanrm.net> wrote:
>
> On Sat, 2 Nov 2019 10:09:27 -0500
> Brian Hansen <dulanic@gmail.com> wrote:
>
> > So I'll need to setup a script to remove C when moving
> > directories.
>
> The only way to un-nocow a +C file is to physically rewrite it into a new
> copy. The +C attribute can be removed on existing files only if they are 0
> bytes in size.
>
> > Unless there is a easier way to deal with this?
>
> Try a different/newer kernel version, as noted in my message all of this
> doesn't seem to matter anymore, and nocow files can now be reflinked (or +C is
> not working anymore and they aren't actually nocow).
>
> --
> With respect,
> Roman

Re: reflink copy now works with nocow?

[Zygo Blaxell]

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

On Sat, Nov 02, 2019 at 07:36:24PM +0500, Roman Mamedov wrote:
> On Sat, 2 Nov 2019 08:49:37 -0500
> Brian Hansen <dulanic@gmail.com> wrote:
> 
> > Hello,
> > 
> > First time i've sent to this group but I am trying to figure out the
> > cause of this. Normal copy is working fine, but then if I use
> > --reflink it says invalid argument. Not sure how to read some of this,
> > but here is the strace.
> > 
> > I'm running kernel v4.15
> > 
> > Here is the full output of strace. I ran a strace on normal copy and
> > most looks similar so I'm not able to figure out much here...
> > 
> > https://pastebin.com/raw/YmQ8FvCH
> 
> At first I was going to say, "oh it's because you are using 'chattr +C', or
> mounted the filesystem as nocow, and reflink copying is prevented by those".
> In fact this article from 2014 confirms that to be the case:
> http://infotinks.com/btrfs-nodatacow-reflink-copies-snapshots/
> 
> But then I tested on my machine, and what used to fail, now works:
> 
>   # mkdir tmp
>   # chattr +C tmp
>   # echo abc > tmp/a
>   # cp -a --reflink=always tmp/a tmp/b
>   # lsattr tmp/
>   ----------------C-- tmp/a
>   ----------------C-- tmp/b
> 
> According to strace, the clone IOCTL succeeds:
> 
> ...
>   openat(AT_FDCWD, "tmp/b", O_WRONLY|O_CREAT|O_EXCL, 0600) = 4
>   fstat(4, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
>   ioctl(4, BTRFS_IOC_CLONE or FICLONE, 3) = 0
> ...
> 
> Same on kernels 4.14.151, 4.14.113 and 4.9.189.
> 
> So I wonder, is setting nocow via 'chattr +C' getting ignored now, or is there
> an improvement that it no longer prevents reflink copying?

reflink copies of nodatacow files should be OK.  'nodatacow' just means
'don't COW *unshared* extents in this file'.  If an extent is shared
by clone, dedupe, or snapshot then it is COW until only one reference
remains.

> -- 
> With respect,
> Roman

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]