Will Btrfs have an official command to "uncow" existing files?

Will Btrfs have an official command to "uncow" existing files?

[Tomokhov Alexander]

Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py

But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.

Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?

Re: Will Btrfs have an official command to "uncow" existing files?

[Duncan]

Tomokhov Alexander posted on Sun, 21 Aug 2016 21:59:36 +0300 as excerpted:

> Btrfs wiki FAQ gives a link to example Python script:
> https://github.com/stsquad/scripts/blob/master/uncow.py
> 
> But such a crucial and fundamental tool must exist in stock btrfs-progs.
> Filesystem with CoW technology at it's core must provide user sufficient
> control over CoW aspects. Running 3rd-party or manually written scripts
> for filesystem properties/metadata manipulation is not convenient, not
> safe and definitely not the way it must be done.

Why?  No script or dedicated tool needed as it's a simple enough process.

Simply:

1. chattr +C (that being nocow) the containing directory.

Then either:

2a. mv the file to and from another filesystem, so it's actually created 
new in the directory and thus inherits the nocow attribute at file 
creation,

or

2b. mv out and then cp the file back into place with --reflink=never, 
again, forcing the file to be created new in the directory, so it 
inherits the nocow attribute at creation,

OR (replacing both steps above)

Create the empty file (using touch or similar), set it nocow, and use cat 
srcfile >> destfile style redirection to fill it, so the file again gets 
the nocow attribute set before it has content, but allowing you to set 
only the file nocow, without setting the containing directory nocow.


Of course there's no exception here to the general case, if you're doing 
the same thing to a whole bunch of files, setting up a script to do it 
may be more efficient than doing it to each one manually one by one, and 
a script could be useful there, but that's a general rule, nothing 
exceptional for btrfs nocow, and a script or fancy tool isn't actually 
required, regardless.


The point being, cow is the default case, and should work /reasonably/ 
well in most cases, certainly well enough so that normal people doing 
normal things shouldn't need to worry about it.  The only people who will 
need to worry about it, therefore, are people worried about the last bit 
of optimization possible to various corner-case use-cases that don't 
match default assumptions very well, and it's precisely these sorts of 
people that are /technical/ enough to be able to understand both why they 
might want nocow (and what the positives and negatives are going to be), 
and how to actually get it.

> Also is it possible (at least in theory) to "uncow" files being
> currently opened in-place? Without the trickery with creation & renaming
> of files or directories. So that running "chattr +C" on a file would be
> sufficient. If possible, is it going to be implemented?

It's software.  Of course it's possible, tho it's also possible the 
negatives make it not worth the trouble.  If the implementation simply 
creates a new file and does a --reflink=never cp in the background when 
the nocow file attribute is set, it may not be worth it.

As to whether it'll ultimately be implemented, I don't know as I'm not a 
dev.  But even if it is ultimately implemented, it might well be five 
years out or longer, because there's simply way more "it'd be nice" btrfs-
related ideas out there than there are devs working on implementations, 
and projecting more than five years out in a software development 
environment like the Linux kernel doesn't make a lot of sense, so five 
years out or longer is likely, but beyond that, nobody really knows.

Add to that the fact that a lot of existing btrfs features took rather 
longer to implement and stabilize than originally projected, and...


Meanwhile, if you aren't already, be aware that the basic concepts of 
snapshots locking in references to existing extents as unchangeable 
(snapshots being a btrfs feature that depends on its cow nature), and 
nocow setting files as rewrite-in-place, basically can't work with each 
other at the concept level, because once a snapshot is taken, extents 
referenced by that snapshot really /cannot/ change until that snapshot is 
deleted, something that can only be true if the extents are copy-on-write.

To work around this btrfs makes the nocow attribute weaker than the 
snapshot locking a particular extent in place, using a process sometimes 
referred to as cow-once or cow1.  When a change is written to a(n 
otherwise) nocow file that has been snapshotted and thus has its extents 
locked in place, the block of the file that is changed will have to be 
cowed, despite the nocow attribute.  However, the nocow attribute is 
kept, and any further changes to that block will be rewritten to the new 
location it was copied to without further cowing of that block, of course 
until the next snapshot locks that location in place as well, at which 
further writes to the same block will of course cow it once more, to a 
third location, which will again remain in place until yet another 
snapshot...

So snapshotting a nocow file means it's no longer absolutely nocow, it's 
now cow1.  If the rewrite rate is higher than the snapshot rate, the nocow 
should still have some effect, but where the snapshot rate is higher than 
the rewrite rate, the effect will be as if the file wasn't nocow at all, 
because each snapshot will lock in the file as it then exists.

It is for this reason that the recommendation is to keep nocow files in a 
dedicated subvolume, so snapshots to the parent subvolume exclude the 
nocow files, and to minimize snapshotting of the nocow subvolume.  As 
long as snapshotting is occurring at all, however, there will be some 
fragmentation over time, but by combining a limited snapshotting rate 
with a reasonable defrag schedule targeting the nowcow files as well, 
fragmentation should at least remain under control.

And of course nocow has other negatives as well.  Setting nocow turns off 
both compression (if you otherwise have compression on) and checksumming, 
thus killing two other major btrfs features, including its realtime file 
integrity validation via checksumming.

So there are definitely negatives to nocow that must be weighed before 
setting it.  But it's worth keeping in mind that all of these features 
are really practical due to cow in the first place, the reason other 
filesystems don't tend to have them, and that while there is definitely a 
tradeoff to cow vs. nocow, setting nocow doesn't turn off features you'd 
have in conventional filesystems anyway, so it's not as if you're losing 
features you'd have if you weren't using btrfs with its cow by default, 
in the first place.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: Will Btrfs have an official command to "uncow" existing files?

[Jeff Mahoney]

[-- Attachment #1.1: Type: text/plain, Size: 1350 bytes --]

On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> 
> But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> 
> Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?

XFS is looking to do this via fallocate using a flag that all file
systems can choose to honor.  Once that lands, it would make sense for
btrfs to use it as well.  The idea is that when you pass the flag in, we
examine the range and CoW anything that has a refcount != 1.

That code hasn't landed yet though.  The last time I saw it posted was
June.  I don't speak with knowledge of the integration plan, but it
might just be queued up for the next merge window now that the reverse
mapping patches have landed in 4.8.

-Jeff

-- 
Jeff Mahoney
SUSE Labs


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: Will Btrfs have an official command to "uncow" existing files?

[Tomokhov Alexander]

Oh, didn't know that XFS is going to have many of Btrfs features and continues to evolve. Thank you for the answer.

22.08.2016, 23:14, "Jeff Mahoney" <jeffm@suse.com>:
> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
>>  Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
>>
>>  But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
>>
>>  Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
>
> XFS is looking to do this via fallocate using a flag that all file
> systems can choose to honor. Once that lands, it would make sense for
> btrfs to use it as well. The idea is that when you pass the flag in, we
> examine the range and CoW anything that has a refcount != 1.
>
> That code hasn't landed yet though. The last time I saw it posted was
> June. I don't speak with knowledge of the integration plan, but it
> might just be queued up for the next merge window now that the reverse
> mapping patches have landed in 4.8.
>
> -Jeff
>
> --
> Jeff Mahoney
> SUSE Labs

Re: Will Btrfs have an official command to "uncow" existing files?

[Darrick J. Wong]

[add Dave and Christoph to cc]

On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> > 
> > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> > 
> > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
> 
> XFS is looking to do this via fallocate using a flag that all file
> systems can choose to honor.  Once that lands, it would make sense for
> btrfs to use it as well.  The idea is that when you pass the flag in, we
> examine the range and CoW anything that has a refcount != 1.

There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
Christoph and Dave felt[1] that the fallocate call didn't need to have
an explicit 'unshare' mode because unsharing shared blocks is
necessary to guarantee that a subsequent write will not ENOSPC.  I
felt that was sufficient justification to withdraw the unshare mode
flag.  If you fallocate the entire length of a shared file on XFS, it
will turn off CoW for that file until you reflink/dedupe it again.

At the time I wondered whether or not the btrfs developers (the list
was cc'd) would pipe up in support of the unshare flag, but nobody
did.  Consequently it remains nonexistent.  Christoph commented a few
months ago about unsharing fallocate over NFS atop XFS blocking for a
long time, though nobody asked for 'unshare' to be reinstated as a
separate fallocate mode, much less a 'don't unshare' flag for regular
fallocate mode.

(FWIW I'm ok with not having to fight for more VFS changes. :))

> That code hasn't landed yet though.  The last time I saw it posted was
> June.  I don't speak with knowledge of the integration plan, but it
> might just be queued up for the next merge window now that the reverse
> mapping patches have landed in 4.8.

I am going to try to land XFS reflink in 4.9; I hope to have an eighth
patchset out for review at the end of the week.

So... if the btrfs folks really want an unshare flag I can trivially
re-add it to the VFS headers and re-enable it in the XFS
implementation <cough> but y'all better speak up now and hammer out an
acceptable definition.  I don't think XFS needs a new flag.

--D

[1] https://www.spinics.net/lists/linux-nfs/msg54740.html

> 
> -Jeff
> 
> -- 
> Jeff Mahoney
> SUSE Labs
> 

Re: Will Btrfs have an official command to "uncow" existing files?

[Tomokhov Alexander]

Thanks for the in-depth answer.

Well, "simple enough process" is still a sequence of steps which must be carefully done. In a proper order with correct parameters depending on environment. It's work with data, data which can be invaluable.
No, really, I'm not a beginner user. I use Arch Linux everyday from 2009, program in different languages and so on. But even a few ordered manual commands for changing file attribute involving quite dangerous "mv" and "cp" (overwrite case) is something very suspicious.

"2a" - step depends on whether another filesystem simply exists, better if it's free, has enough space, supports same file permission features, etc. Requires time to figure out these conditions, not suitable for all systems.

"2b" - step with "mv out". Out to where? What if the file with the same name already exists in a destination directory you "mv out". Not reliable. Ok, need to create temporary directory. Where, how to call it then - involves conditional checks performed by user.

Similarly creation of empty file should also satisfy the condition that it's name is unique in the directory.

Additionally all existing ways of "uncow" require manual free space check beforehand. User must control and monitor if the file is currently not opened. I'm sure I missed something else.
These all are problems that are unrelated to file attributes itself, but user must think of them for some reason. An official specialized tool could automatically track all these conditions, perform the right sequence of actions and report to user results.

Yes I do take into consideration that there are situations when "uncow" cannot be actually applied to a file for the reasons you described. No snapshots atm in my case and, for example, I have firefox sqlite database file with 900+ extents on a rotational disk. I wouldn't say it's noticeable, but at least desire the number of extents not to increase further so that I won't notice it ever. I admit that Btrfs may defragment it, but may not. Sometimes we need a more controllable approach.

22.08.2016, 05:00, "Duncan" <1i5t5.duncan@cox.net>:
> Tomokhov Alexander posted on Sun, 21 Aug 2016 21:59:36 +0300 as excerpted:
>
>>  Btrfs wiki FAQ gives a link to example Python script:
>>  https://github.com/stsquad/scripts/blob/master/uncow.py
>>
>>  But such a crucial and fundamental tool must exist in stock btrfs-progs.
>>  Filesystem with CoW technology at it's core must provide user sufficient
>>  control over CoW aspects. Running 3rd-party or manually written scripts
>>  for filesystem properties/metadata manipulation is not convenient, not
>>  safe and definitely not the way it must be done.
>
> Why? No script or dedicated tool needed as it's a simple enough process.
>
> Simply:
>
> 1. chattr +C (that being nocow) the containing directory.
>
> Then either:
>
> 2a. mv the file to and from another filesystem, so it's actually created
> new in the directory and thus inherits the nocow attribute at file
> creation,
>
> or
>
> 2b. mv out and then cp the file back into place with --reflink=never,
> again, forcing the file to be created new in the directory, so it
> inherits the nocow attribute at creation,
>
> OR (replacing both steps above)
>
> Create the empty file (using touch or similar), set it nocow, and use cat
> srcfile >> destfile style redirection to fill it, so the file again gets
> the nocow attribute set before it has content, but allowing you to set
> only the file nocow, without setting the containing directory nocow.
>
> Of course there's no exception here to the general case, if you're doing
> the same thing to a whole bunch of files, setting up a script to do it
> may be more efficient than doing it to each one manually one by one, and
> a script could be useful there, but that's a general rule, nothing
> exceptional for btrfs nocow, and a script or fancy tool isn't actually
> required, regardless.
>
> The point being, cow is the default case, and should work /reasonably/
> well in most cases, certainly well enough so that normal people doing
> normal things shouldn't need to worry about it. The only people who will
> need to worry about it, therefore, are people worried about the last bit
> of optimization possible to various corner-case use-cases that don't
> match default assumptions very well, and it's precisely these sorts of
> people that are /technical/ enough to be able to understand both why they
> might want nocow (and what the positives and negatives are going to be),
> and how to actually get it.
>
>>  Also is it possible (at least in theory) to "uncow" files being
>>  currently opened in-place? Without the trickery with creation & renaming
>>  of files or directories. So that running "chattr +C" on a file would be
>>  sufficient. If possible, is it going to be implemented?
>
> It's software. Of course it's possible, tho it's also possible the
> negatives make it not worth the trouble. If the implementation simply
> creates a new file and does a --reflink=never cp in the background when
> the nocow file attribute is set, it may not be worth it.
>
> As to whether it'll ultimately be implemented, I don't know as I'm not a
> dev. But even if it is ultimately implemented, it might well be five
> years out or longer, because there's simply way more "it'd be nice" btrfs-
> related ideas out there than there are devs working on implementations,
> and projecting more than five years out in a software development
> environment like the Linux kernel doesn't make a lot of sense, so five
> years out or longer is likely, but beyond that, nobody really knows.
>
> Add to that the fact that a lot of existing btrfs features took rather
> longer to implement and stabilize than originally projected, and...
>
> Meanwhile, if you aren't already, be aware that the basic concepts of
> snapshots locking in references to existing extents as unchangeable
> (snapshots being a btrfs feature that depends on its cow nature), and
> nocow setting files as rewrite-in-place, basically can't work with each
> other at the concept level, because once a snapshot is taken, extents
> referenced by that snapshot really /cannot/ change until that snapshot is
> deleted, something that can only be true if the extents are copy-on-write.
>
> To work around this btrfs makes the nocow attribute weaker than the
> snapshot locking a particular extent in place, using a process sometimes
> referred to as cow-once or cow1. When a change is written to a(n
> otherwise) nocow file that has been snapshotted and thus has its extents
> locked in place, the block of the file that is changed will have to be
> cowed, despite the nocow attribute. However, the nocow attribute is
> kept, and any further changes to that block will be rewritten to the new
> location it was copied to without further cowing of that block, of course
> until the next snapshot locks that location in place as well, at which
> further writes to the same block will of course cow it once more, to a
> third location, which will again remain in place until yet another
> snapshot...
>
> So snapshotting a nocow file means it's no longer absolutely nocow, it's
> now cow1. If the rewrite rate is higher than the snapshot rate, the nocow
> should still have some effect, but where the snapshot rate is higher than
> the rewrite rate, the effect will be as if the file wasn't nocow at all,
> because each snapshot will lock in the file as it then exists.
>
> It is for this reason that the recommendation is to keep nocow files in a
> dedicated subvolume, so snapshots to the parent subvolume exclude the
> nocow files, and to minimize snapshotting of the nocow subvolume. As
> long as snapshotting is occurring at all, however, there will be some
> fragmentation over time, but by combining a limited snapshotting rate
> with a reasonable defrag schedule targeting the nowcow files as well,
> fragmentation should at least remain under control.
>
> And of course nocow has other negatives as well. Setting nocow turns off
> both compression (if you otherwise have compression on) and checksumming,
> thus killing two other major btrfs features, including its realtime file
> integrity validation via checksumming.
>
> So there are definitely negatives to nocow that must be weighed before
> setting it. But it's worth keeping in mind that all of these features
> are really practical due to cow in the first place, the reason other
> filesystems don't tend to have them, and that while there is definitely a
> tradeoff to cow vs. nocow, setting nocow doesn't turn off features you'd
> have in conventional filesystems anyway, so it's not as if you're losing
> features you'd have if you weren't using btrfs with its cow by default,
> in the first place.
>
> --
> Duncan - List replies preferred. No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master." Richard Stallman
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Will Btrfs have an official command to "uncow" existing files?

[Chris Murphy]

On Mon, Aug 22, 2016 at 5:06 PM, Darrick J. Wong
<darrick.wong@oracle.com> wrote:
> [add Dave and Christoph to cc]
>
> On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
>> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
>> > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
>> >
>> > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
>> >
>> > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
>>
>> XFS is looking to do this via fallocate using a flag that all file
>> systems can choose to honor.  Once that lands, it would make sense for
>> btrfs to use it as well.  The idea is that when you pass the flag in, we
>> examine the range and CoW anything that has a refcount != 1.
>
> There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> Christoph and Dave felt[1] that the fallocate call didn't need to have
> an explicit 'unshare' mode because unsharing shared blocks is
> necessary to guarantee that a subsequent write will not ENOSPC.  I
> felt that was sufficient justification to withdraw the unshare mode
> flag.  If you fallocate the entire length of a shared file on XFS, it
> will turn off CoW for that file until you reflink/dedupe it again.
>
> At the time I wondered whether or not the btrfs developers (the list
> was cc'd) would pipe up in support of the unshare flag, but nobody
> did.  Consequently it remains nonexistent.  Christoph commented a few
> months ago about unsharing fallocate over NFS atop XFS blocking for a
> long time, though nobody asked for 'unshare' to be reinstated as a
> separate fallocate mode, much less a 'don't unshare' flag for regular
> fallocate mode.
>
> (FWIW I'm ok with not having to fight for more VFS changes. :))
>
>> That code hasn't landed yet though.  The last time I saw it posted was
>> June.  I don't speak with knowledge of the integration plan, but it
>> might just be queued up for the next merge window now that the reverse
>> mapping patches have landed in 4.8.
>
> I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> patchset out for review at the end of the week.
>
> So... if the btrfs folks really want an unshare flag I can trivially
> re-add it to the VFS headers and re-enable it in the XFS
> implementation <cough> but y'all better speak up now and hammer out an
> acceptable definition.  I don't think XFS needs a new flag.

Use case wise I can't think of why I'd want to do unshare. There is a
use case for wanting to set nocow after the fact. I have no idea what
complexity is added on the Btrfs side for either operation, it seems
like at the least to set it, data csum needs a way to be ignored or
removed; and conversely to unset nocow it's a question whether that
means the file should have csum's computed, strictly speaking I guess
you could have cow without datacsum.


-- 
Chris Murphy

Re: Will Btrfs have an official command to "uncow" existing files?

[Dave Chinner]

On Mon, Aug 22, 2016 at 04:06:13PM -0700, Darrick J. Wong wrote:
> [add Dave and Christoph to cc]
> 
> On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
> > On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> > > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> > > 
> > > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> > > 
> > > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
> > 
> > XFS is looking to do this via fallocate using a flag that all file
> > systems can choose to honor.  Once that lands, it would make sense for
> > btrfs to use it as well.  The idea is that when you pass the flag in, we
> > examine the range and CoW anything that has a refcount != 1.
> 
> There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> Christoph and Dave felt[1] that the fallocate call didn't need to have
> an explicit 'unshare' mode because unsharing shared blocks is
> necessary to guarantee that a subsequent write will not ENOSPC.  I
> felt that was sufficient justification to withdraw the unshare mode
> flag.  If you fallocate the entire length of a shared file on XFS, it
> will turn off CoW for that file until you reflink/dedupe it again.

>From the XFS POV that's all good...

> At the time I wondered whether or not the btrfs developers (the list
> was cc'd) would pipe up in support of the unshare flag, but nobody
> did.  Consequently it remains nonexistent.  Christoph commented a few
> months ago about unsharing fallocate over NFS atop XFS blocking for a
> long time, though nobody asked for 'unshare' to be reinstated as a
> separate fallocate mode, much less a 'don't unshare' flag for regular
> fallocate mode.

If there are other use cases, then we can easily implement it in
XFS. However, let's not overload the XFS reflink code with things
other fs devs have once said "that'd be nice to do"....

> (FWIW I'm ok with not having to fight for more VFS changes. :))
> 
> > That code hasn't landed yet though.  The last time I saw it posted was
> > June.  I don't speak with knowledge of the integration plan, but it
> > might just be queued up for the next merge window now that the reverse
> > mapping patches have landed in 4.8.
> 
> I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> patchset out for review at the end of the week.
> 
> So... if the btrfs folks really want an unshare flag I can trivially
> re-add it to the VFS headers and re-enable it in the XFS
> implementation <cough> but y'all better speak up now and hammer out an
> acceptable definition.  I don't think XFS needs a new flag.

It's not urgent - it can be added at any time so I'd say it
something we should ignore on the XFS side of things until someone
actually requires an explicit "unshare" operation for another
filesystem....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

Re: Will Btrfs have an official command to "uncow" existing files?

[Austin S. Hemmelgarn]

On 2016-08-22 22:43, Chris Murphy wrote:
> On Mon, Aug 22, 2016 at 5:06 PM, Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
>> [add Dave and Christoph to cc]
>>
>> On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
>>> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
>>>> Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
>>>>
>>>> But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
>>>>
>>>> Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
>>>
>>> XFS is looking to do this via fallocate using a flag that all file
>>> systems can choose to honor.  Once that lands, it would make sense for
>>> btrfs to use it as well.  The idea is that when you pass the flag in, we
>>> examine the range and CoW anything that has a refcount != 1.
>>
>> There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
>> Christoph and Dave felt[1] that the fallocate call didn't need to have
>> an explicit 'unshare' mode because unsharing shared blocks is
>> necessary to guarantee that a subsequent write will not ENOSPC.  I
>> felt that was sufficient justification to withdraw the unshare mode
>> flag.  If you fallocate the entire length of a shared file on XFS, it
>> will turn off CoW for that file until you reflink/dedupe it again.
>>
>> At the time I wondered whether or not the btrfs developers (the list
>> was cc'd) would pipe up in support of the unshare flag, but nobody
>> did.  Consequently it remains nonexistent.  Christoph commented a few
>> months ago about unsharing fallocate over NFS atop XFS blocking for a
>> long time, though nobody asked for 'unshare' to be reinstated as a
>> separate fallocate mode, much less a 'don't unshare' flag for regular
>> fallocate mode.
>>
>> (FWIW I'm ok with not having to fight for more VFS changes. :))
>>
>>> That code hasn't landed yet though.  The last time I saw it posted was
>>> June.  I don't speak with knowledge of the integration plan, but it
>>> might just be queued up for the next merge window now that the reverse
>>> mapping patches have landed in 4.8.
>>
>> I am going to try to land XFS reflink in 4.9; I hope to have an eighth
>> patchset out for review at the end of the week.
>>
>> So... if the btrfs folks really want an unshare flag I can trivially
>> re-add it to the VFS headers and re-enable it in the XFS
>> implementation <cough> but y'all better speak up now and hammer out an
>> acceptable definition.  I don't think XFS needs a new flag.
>
> Use case wise I can't think of why I'd want to do unshare. There is a
> use case for wanting to set nocow after the fact. I have no idea what
> complexity is added on the Btrfs side for either operation, it seems
> like at the least to set it, data csum needs a way to be ignored or
> removed; and conversely to unset nocow it's a question whether that
> means the file should have csum's computed, strictly speaking I guess
> you could have cow without datacsum.
The primary use case I can think of is making sure that that particular 
file has enough space to store all it's data on disk.  COW makes it 
somewhat non-deterministic whether or not a write will fail, which is 
the other reason that NOCOW is useful.  Some day, we really should have 
the ability to set NOCOW on arbitrary files which already have data in 
them, and being able to forcibly unshare all extents in a file would be 
an important part of being able to do that.

This all brings up a slightly bigger question though.  How exactly does 
fallocate work on BTRFS?  If we're not forcibly doing a COW of all 
shared extents in a file when fallocate is called over the whole file, 
then it's debatable whether we actually provide the semantics expected 
of fallocate.  Strictly speaking, without some complicated internal code 
and some kind of extra information in the FS< it's not possible for us 
to honor fallocate correctly at all except when dealing with new 
allocations or NOCOW files, because we don't reserve temporary space for 
COW to happen in the data chunks.

Re: Will Btrfs have an official command to "uncow" existing files?

[Jeff Mahoney]

[-- Attachment #1.1: Type: text/plain, Size: 3867 bytes --]

On 8/22/16 7:06 PM, Darrick J. Wong wrote:
> [add Dave and Christoph to cc]
> 
> On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
>> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
>>> Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
>>>
>>> But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
>>>
>>> Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
>>
>> XFS is looking to do this via fallocate using a flag that all file
>> systems can choose to honor.  Once that lands, it would make sense for
>> btrfs to use it as well.  The idea is that when you pass the flag in, we
>> examine the range and CoW anything that has a refcount != 1.
> 
> There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> Christoph and Dave felt[1] that the fallocate call didn't need to have
> an explicit 'unshare' mode because unsharing shared blocks is
> necessary to guarantee that a subsequent write will not ENOSPC.  I
> felt that was sufficient justification to withdraw the unshare mode
> flag.  If you fallocate the entire length of a shared file on XFS, it
> will turn off CoW for that file until you reflink/dedupe it again.

Is that a flag or just that it's reverting to "normal" XFS operation?
We have a nocow flag for btrfs, but it's more like nocow* because it's
still possible to create new references to the extents and those must be
CoW'd later.  I think that's about all we can offer since the nocow flag
set otherwise would imply that /every/ snapshot does a full copy of
anything marked nocow and I don't think that's the expectation either.

> At the time I wondered whether or not the btrfs developers (the list
> was cc'd) would pipe up in support of the unshare flag, but nobody
> did.  Consequently it remains nonexistent.  Christoph commented a few
> months ago about unsharing fallocate over NFS atop XFS blocking for a
> long time, though nobody asked for 'unshare' to be reinstated as a
> separate fallocate mode, much less a 'don't unshare' flag for regular
> fallocate mode.
> 
> (FWIW I'm ok with not having to fight for more VFS changes. :))

Agreed. :)

>> That code hasn't landed yet though.  The last time I saw it posted was
>> June.  I don't speak with knowledge of the integration plan, but it
>> might just be queued up for the next merge window now that the reverse
>> mapping patches have landed in 4.8.
> 
> I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> patchset out for review at the end of the week.
> 
> So... if the btrfs folks really want an unshare flag I can trivially
> re-add it to the VFS headers and re-enable it in the XFS
> implementation <cough> but y'all better speak up now and hammer out an
> acceptable definition.  I don't think XFS needs a new flag.

Thanks for the explanation, Darrick.  I'm not advocating for a flag.
That was just the last state of the implementation as I remember it.  I
missed the discussion turning to not needing it at all.

I suppose the only thing missing, and this applies to both XFS and a
future btrfs implementation, is documentation for the user explaining
that the guarantees fallocate makes about ENOSPC not failing are voided
by operations that can re-share extents.

-Jeff

-- 
Jeff Mahoney
SUSE Labs



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 827 bytes --]

Re: Will Btrfs have an official command to "uncow" existing files?

[Darrick J. Wong]

On Tue, Aug 23, 2016 at 08:48:20PM -0400, Jeff Mahoney wrote:
> On 8/22/16 7:06 PM, Darrick J. Wong wrote:
> > [add Dave and Christoph to cc]
> > 
> > On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
> >> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> >>> Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> >>>
> >>> But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> >>>
> >>> Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
> >>
> >> XFS is looking to do this via fallocate using a flag that all file
> >> systems can choose to honor.  Once that lands, it would make sense for
> >> btrfs to use it as well.  The idea is that when you pass the flag in, we
> >> examine the range and CoW anything that has a refcount != 1.
> > 
> > There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> > Christoph and Dave felt[1] that the fallocate call didn't need to have
> > an explicit 'unshare' mode because unsharing shared blocks is
> > necessary to guarantee that a subsequent write will not ENOSPC.  I
> > felt that was sufficient justification to withdraw the unshare mode
> > flag.  If you fallocate the entire length of a shared file on XFS, it
> > will turn off CoW for that file until you reflink/dedupe it again.
> 
> Is that a flag or just that it's reverting to "normal" XFS operation?

Inodes with shared extents have a 'reflink' flag to indicate that
somewhere in the data fork are (potentially) blocks that need CoW.
The 'fallocate the entire file' code turns off the flag after which
all writes revert to regular XFS overwrite behavior.

> We have a nocow flag for btrfs, but it's more like nocow* because it's
> still possible to create new references to the extents and those must be
> CoW'd later.  I think that's about all we can offer since the nocow flag
> set otherwise would imply that /every/ snapshot does a full copy of
> anything marked nocow and I don't think that's the expectation either.

Copy-everything is certainly not what it /does/. :)

> > At the time I wondered whether or not the btrfs developers (the list
> > was cc'd) would pipe up in support of the unshare flag, but nobody
> > did.  Consequently it remains nonexistent.  Christoph commented a few
> > months ago about unsharing fallocate over NFS atop XFS blocking for a
> > long time, though nobody asked for 'unshare' to be reinstated as a
> > separate fallocate mode, much less a 'don't unshare' flag for regular
> > fallocate mode.
> > 
> > (FWIW I'm ok with not having to fight for more VFS changes. :))
> 
> Agreed. :)
> 
> >> That code hasn't landed yet though.  The last time I saw it posted was
> >> June.  I don't speak with knowledge of the integration plan, but it
> >> might just be queued up for the next merge window now that the reverse
> >> mapping patches have landed in 4.8.
> > 
> > I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> > patchset out for review at the end of the week.
> > 
> > So... if the btrfs folks really want an unshare flag I can trivially
> > re-add it to the VFS headers and re-enable it in the XFS
> > implementation <cough> but y'all better speak up now and hammer out an
> > acceptable definition.  I don't think XFS needs a new flag.
> 
> Thanks for the explanation, Darrick.  I'm not advocating for a flag.
> That was just the last state of the implementation as I remember it.  I
> missed the discussion turning to not needing it at all.
> 
> I suppose the only thing missing, and this applies to both XFS and a
> future btrfs implementation, is documentation for the user explaining
> that the guarantees fallocate makes about ENOSPC not failing are voided
> by operations that can re-share extents.

Indeed, updating the manpage for fallocate (and dedupe) is still on my list.

--D

> 
> -Jeff
> 
> -- 
> Jeff Mahoney
> SUSE Labs
> 
> 

Re: Will Btrfs have an official command to "uncow" existing files?

[Omar Sandoval]

On Mon, Aug 22, 2016 at 08:43:18PM -0600, Chris Murphy wrote:
> On Mon, Aug 22, 2016 at 5:06 PM, Darrick J. Wong
> <darrick.wong@oracle.com> wrote:
> > [add Dave and Christoph to cc]
> >
> > On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
> >> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> >> > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> >> >
> >> > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> >> >
> >> > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
> >>
> >> XFS is looking to do this via fallocate using a flag that all file
> >> systems can choose to honor.  Once that lands, it would make sense for
> >> btrfs to use it as well.  The idea is that when you pass the flag in, we
> >> examine the range and CoW anything that has a refcount != 1.
> >
> > There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> > Christoph and Dave felt[1] that the fallocate call didn't need to have
> > an explicit 'unshare' mode because unsharing shared blocks is
> > necessary to guarantee that a subsequent write will not ENOSPC.  I
> > felt that was sufficient justification to withdraw the unshare mode
> > flag.  If you fallocate the entire length of a shared file on XFS, it
> > will turn off CoW for that file until you reflink/dedupe it again.
> >
> > At the time I wondered whether or not the btrfs developers (the list
> > was cc'd) would pipe up in support of the unshare flag, but nobody
> > did.  Consequently it remains nonexistent.  Christoph commented a few
> > months ago about unsharing fallocate over NFS atop XFS blocking for a
> > long time, though nobody asked for 'unshare' to be reinstated as a
> > separate fallocate mode, much less a 'don't unshare' flag for regular
> > fallocate mode.
> >
> > (FWIW I'm ok with not having to fight for more VFS changes. :))
> >
> >> That code hasn't landed yet though.  The last time I saw it posted was
> >> June.  I don't speak with knowledge of the integration plan, but it
> >> might just be queued up for the next merge window now that the reverse
> >> mapping patches have landed in 4.8.
> >
> > I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> > patchset out for review at the end of the week.
> >
> > So... if the btrfs folks really want an unshare flag I can trivially
> > re-add it to the VFS headers and re-enable it in the XFS
> > implementation <cough> but y'all better speak up now and hammer out an
> > acceptable definition.  I don't think XFS needs a new flag.
> 
> Use case wise I can't think of why I'd want to do unshare. There is a
> use case for wanting to set nocow after the fact. I have no idea what
> complexity is added on the Btrfs side for either operation, it seems
> like at the least to set it, data csum needs a way to be ignored or
> removed; and conversely to unset nocow it's a question whether that
> means the file should have csum's computed, strictly speaking I guess
> you could have cow without datacsum.

One use case is for swapfile support on Btrfs -- I implemented it with
the requirement that the file was nocow with no shared extents. I think
there was some discussion about having the swapon operation do that
unshare, but I decided against that [1]. (I should take a look at
reviving that patch series.)

Darrick, what's XFS doing for reflink + swap files?

1: http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg785536.html

-- 
Omar

Re: Will Btrfs have an official command to "uncow" existing files?

[Darrick J. Wong]

On Wed, Aug 24, 2016 at 11:34:27AM -0700, Omar Sandoval wrote:
> On Mon, Aug 22, 2016 at 08:43:18PM -0600, Chris Murphy wrote:
> > On Mon, Aug 22, 2016 at 5:06 PM, Darrick J. Wong
> > <darrick.wong@oracle.com> wrote:
> > > [add Dave and Christoph to cc]
> > >
> > > On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
> > >> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> > >> > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> > >> >
> > >> > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> > >> >
> > >> > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
> > >>
> > >> XFS is looking to do this via fallocate using a flag that all file
> > >> systems can choose to honor.  Once that lands, it would make sense for
> > >> btrfs to use it as well.  The idea is that when you pass the flag in, we
> > >> examine the range and CoW anything that has a refcount != 1.
> > >
> > > There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> > > Christoph and Dave felt[1] that the fallocate call didn't need to have
> > > an explicit 'unshare' mode because unsharing shared blocks is
> > > necessary to guarantee that a subsequent write will not ENOSPC.  I
> > > felt that was sufficient justification to withdraw the unshare mode
> > > flag.  If you fallocate the entire length of a shared file on XFS, it
> > > will turn off CoW for that file until you reflink/dedupe it again.
> > >
> > > At the time I wondered whether or not the btrfs developers (the list
> > > was cc'd) would pipe up in support of the unshare flag, but nobody
> > > did.  Consequently it remains nonexistent.  Christoph commented a few
> > > months ago about unsharing fallocate over NFS atop XFS blocking for a
> > > long time, though nobody asked for 'unshare' to be reinstated as a
> > > separate fallocate mode, much less a 'don't unshare' flag for regular
> > > fallocate mode.
> > >
> > > (FWIW I'm ok with not having to fight for more VFS changes. :))
> > >
> > >> That code hasn't landed yet though.  The last time I saw it posted was
> > >> June.  I don't speak with knowledge of the integration plan, but it
> > >> might just be queued up for the next merge window now that the reverse
> > >> mapping patches have landed in 4.8.
> > >
> > > I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> > > patchset out for review at the end of the week.
> > >
> > > So... if the btrfs folks really want an unshare flag I can trivially
> > > re-add it to the VFS headers and re-enable it in the XFS
> > > implementation <cough> but y'all better speak up now and hammer out an
> > > acceptable definition.  I don't think XFS needs a new flag.
> > 
> > Use case wise I can't think of why I'd want to do unshare. There is a
> > use case for wanting to set nocow after the fact. I have no idea what
> > complexity is added on the Btrfs side for either operation, it seems
> > like at the least to set it, data csum needs a way to be ignored or
> > removed; and conversely to unset nocow it's a question whether that
> > means the file should have csum's computed, strictly speaking I guess
> > you could have cow without datacsum.
> 
> One use case is for swapfile support on Btrfs -- I implemented it with
> the requirement that the file was nocow with no shared extents. I think
> there was some discussion about having the swapon operation do that
> unshare, but I decided against that [1]. (I should take a look at
> reviving that patch series.)
> 
> Darrick, what's XFS doing for reflink + swap files?
> 
> 1: http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg785536.html

We neither allow swapon() for file with shared extents, nor reflinking/deduping
files currently being used as swap.

--D

> 
> -- 
> Omar

Re: Will Btrfs have an official command to "uncow" existing files?

[Omar Sandoval]

On Wed, Aug 24, 2016 at 03:42:58PM -0700, Darrick J. Wong wrote:
> On Wed, Aug 24, 2016 at 11:34:27AM -0700, Omar Sandoval wrote:
> > On Mon, Aug 22, 2016 at 08:43:18PM -0600, Chris Murphy wrote:
> > > On Mon, Aug 22, 2016 at 5:06 PM, Darrick J. Wong
> > > <darrick.wong@oracle.com> wrote:
> > > > [add Dave and Christoph to cc]
> > > >
> > > > On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote:
> > > >> On 8/21/16 2:59 PM, Tomokhov Alexander wrote:
> > > >> > Btrfs wiki FAQ gives a link to example Python script: https://github.com/stsquad/scripts/blob/master/uncow.py
> > > >> >
> > > >> > But such a crucial and fundamental tool must exist in stock btrfs-progs. Filesystem with CoW technology at it's core must provide user sufficient control over CoW aspects. Running 3rd-party or manually written scripts for filesystem properties/metadata manipulation is not convenient, not safe and definitely not the way it must be done.
> > > >> >
> > > >> > Also is it possible (at least in theory) to "uncow" files being currently opened in-place? Without the trickery with creation & renaming of files or directories. So that running "chattr +C" on a file would be sufficient. If possible, is it going to be implemented?
> > > >>
> > > >> XFS is looking to do this via fallocate using a flag that all file
> > > >> systems can choose to honor.  Once that lands, it would make sense for
> > > >> btrfs to use it as well.  The idea is that when you pass the flag in, we
> > > >> examine the range and CoW anything that has a refcount != 1.
> > > >
> > > > There /was/ a flag to do that -- FALLOC_FL_UNSHARE_RANGE.  However,
> > > > Christoph and Dave felt[1] that the fallocate call didn't need to have
> > > > an explicit 'unshare' mode because unsharing shared blocks is
> > > > necessary to guarantee that a subsequent write will not ENOSPC.  I
> > > > felt that was sufficient justification to withdraw the unshare mode
> > > > flag.  If you fallocate the entire length of a shared file on XFS, it
> > > > will turn off CoW for that file until you reflink/dedupe it again.
> > > >
> > > > At the time I wondered whether or not the btrfs developers (the list
> > > > was cc'd) would pipe up in support of the unshare flag, but nobody
> > > > did.  Consequently it remains nonexistent.  Christoph commented a few
> > > > months ago about unsharing fallocate over NFS atop XFS blocking for a
> > > > long time, though nobody asked for 'unshare' to be reinstated as a
> > > > separate fallocate mode, much less a 'don't unshare' flag for regular
> > > > fallocate mode.
> > > >
> > > > (FWIW I'm ok with not having to fight for more VFS changes. :))
> > > >
> > > >> That code hasn't landed yet though.  The last time I saw it posted was
> > > >> June.  I don't speak with knowledge of the integration plan, but it
> > > >> might just be queued up for the next merge window now that the reverse
> > > >> mapping patches have landed in 4.8.
> > > >
> > > > I am going to try to land XFS reflink in 4.9; I hope to have an eighth
> > > > patchset out for review at the end of the week.
> > > >
> > > > So... if the btrfs folks really want an unshare flag I can trivially
> > > > re-add it to the VFS headers and re-enable it in the XFS
> > > > implementation <cough> but y'all better speak up now and hammer out an
> > > > acceptable definition.  I don't think XFS needs a new flag.
> > > 
> > > Use case wise I can't think of why I'd want to do unshare. There is a
> > > use case for wanting to set nocow after the fact. I have no idea what
> > > complexity is added on the Btrfs side for either operation, it seems
> > > like at the least to set it, data csum needs a way to be ignored or
> > > removed; and conversely to unset nocow it's a question whether that
> > > means the file should have csum's computed, strictly speaking I guess
> > > you could have cow without datacsum.
> > 
> > One use case is for swapfile support on Btrfs -- I implemented it with
> > the requirement that the file was nocow with no shared extents. I think
> > there was some discussion about having the swapon operation do that
> > unshare, but I decided against that [1]. (I should take a look at
> > reviving that patch series.)
> > 
> > Darrick, what's XFS doing for reflink + swap files?
> > 
> > 1: http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg785536.html
> 
> We neither allow swapon() for file with shared extents, nor reflinking/deduping
> files currently being used as swap.
> 
> --D

Perfect, so that's consistent with the behavior from my series.

-- 
Omar

Re: Will Btrfs have an official command to "uncow" existing files?

[Patrik Lundquist]

On Mon, 30 Jan 2023 at 17:39, Patrik Lundquist
<patrik.lundquist@gmail.com> wrote:
>
> I threw together https://github.com/pLuster/btrfs-uncow and even tried
> interrupting and continuing a half finished copy once, which worked
> fine. A huge drawback is that files with holes will grow.
>
> It's not well tested and I don't know if it's very useful. Copying
> zeros should be smarter. Open for suggestions.

Goffredo was quick to add support for file holes. Thanks!

Re: Will Btrfs have an official command to "uncow" existing files?

[Patrik Lundquist]

On Mon, 23 Jan 2023 at 01:36, Zygo Blaxell
<ce3g8jdj@umail.furryterror.org> wrote:
>
> On Sun, Jan 22, 2023 at 09:27:33PM +0100, Goffredo Baroncelli wrote:
> > On 22/01/2023 12.41, Cerem Cem ASLAN wrote:
> > > Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html
> > >
> > > The problem with the "chattr +C ..., move back and forth" approach is
> > > that the VM folder is about 300GB and I have ~100GB of free space,
> > > plus,
> >
> > This can be solvable: it should be possible to make a tool that for
> > each unit of copy (eg each 1 GB) does:
> > - copy the data from the COW file to the NOCOW file
> > - remove the data copied from the NOCOW file (using fallocate+FALLOC_FL_PUNCH_HOLE)
> >
> > So you can avoid to have two FULL copy of the same file (in pseudo code:
> >
> > block_size = 1024*1024*1024;
> > while (pos < file_len) {
> >       l = min(block_size, file_len - pos);
> >       len_copied = copy(srcfd, dstfd, pos, l);
> >       fallocate(srcfd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, pos, l);
> >
> >       pos += l;
> > }
> >
> >
> > end pseudo code)
> >
> > I don't know if there is an algorithm that prevent the data loss in case of
> > an interruption of the copy. May be that it exists... We need a file where
> > we could log the status.
>
> Start at the end of the file and work backwards, making the source
> file shorter with truncate.  The size of the original file provides a
> built-in checkpoint that you can resume from, at worst, copying the last
> 1GB of the file again after resuming.  There's no danger of accidentally
> copying a bunch of zeros that would be left behind by PUNCH_HOLE.
>
> Don't forget to open the dst file without O_TRUNC, and call fsync()
> (or ideally sync() to force a full commit and avoid possible log tree
> replay issues) between writing the copy and truncating the original.

I threw together https://github.com/pLuster/btrfs-uncow and even tried
interrupting and continuing a half finished copy once, which worked
fine. A huge drawback is that files with holes will grow.

It's not well tested and I don't know if it's very useful. Copying
zeros should be smarter. Open for suggestions.

Re: Will Btrfs have an official command to "uncow" existing files?

[Zygo Blaxell]

On Sun, Jan 22, 2023 at 11:17:49PM -0800, Christoph Hellwig wrote:
> On Sun, Jan 22, 2023 at 02:41:22PM +0300, Cerem Cem ASLAN wrote:
> > Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html
> > 
> > The problem with the "chattr +C ..., move back and forth" approach is
> > that the VM folder is about 300GB and I have ~100GB of free space,
> > plus, I have multiple copies which will require that 300GB to
> > re-transfer after deleting all previous snapshots (because there is no
> > enough free space on those backup hard disks).
> > 
> > So, we really need to set the NoCow attribute for the existing files.
> > 
> > Should we currently use a separate partition for VMs and mount it with
> > nodatacow option to avoid that issue?
> 
> So, Linux for a while now has the FALLOC_FL_UNSHARE_RANGE flag to
> fallocate to unshare previously shared extents.  It still lacks an
> implementation for btrfs, but it seems to be the interface that you
> want.

Users can already get unshared extents with the btrfs DEFRAG_RANGE ioctl,
but in a datacow file, the result is an unshared copy-on-write extent,
not a write-in-place extent.  If this solved the original problem,
it would be in a FAQ somewhere.

The btrfs datacow attribute is in the inode, not the extent.  Extents in
a datacow file are copy-on-write regardless of extent sharing (ZERO_RANGE
allocations are a temporary exception, but eventually revert back to
copy-on-write).  Extents in a nodatacow file are copy-on-write if the
extents are shared at write time; otherwise, they are write-in-place.

If btrfs is changed to move the nodatacow status into the extent
reference, it would allow individual extents to have independently
settable datacow/nodatacow bits, and the inode attribute would be
merely a hint for what value to use when creating new extents.  If that
was done, UNSHARE_RANGE could have the expected effect on allocation,
but an unexpected side-effect of disabling data csums at the same time
(datasum relies on datacow, setting nodatacow also sets nodatasum).

Better to not implement UNSHARE_RANGE at all than to have an
implementation with unexpected and unpleasant side-effects.  Or add an
extra flag to the fallocate interface to indicate whether a downgrade in
data integrity is intended and permitted, so it won't happen unexpectedly.

The experience of ZERO_RANGE on btrfs can be informative.  Most of the
above can also be said about btrfs support for ZERO_RANGE on datacow
files too (ZERO_RANGE on nodatacow files is sane, but OP's problem is how
to make an existing datacow file become nodatacow).  ZERO_RANGE is also
an inode attribute on btrfs (called PREALLOC), and for datacow files it
causes more problems than ZERO_RANGE was ever intended to solve, while
not implementing the allocation guarantees promised in the ZERO_RANGE
documentation (ENOSPC is still possible, and potentially more likely,
after fallocate with ZERO_RANGE on a datacow file).  As a bonus feature,
ZERO_RANGE cannot be turned off in an inode once it is set, disrupting
other features like compression.  With per-extent datacow attributes,
at least the damage can be limited to the offset and length arguments
of fallocate(2), instead of tainting the entire inode forever.

So I guess that makes two btrfs problems that can be solved by the same
modification of btrfs on-disk structures.

Re: Will Btrfs have an official command to "uncow" existing files?

[Christoph Hellwig]

On Sun, Jan 22, 2023 at 02:41:22PM +0300, Cerem Cem ASLAN wrote:
> Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html
> 
> The problem with the "chattr +C ..., move back and forth" approach is
> that the VM folder is about 300GB and I have ~100GB of free space,
> plus, I have multiple copies which will require that 300GB to
> re-transfer after deleting all previous snapshots (because there is no
> enough free space on those backup hard disks).
> 
> So, we really need to set the NoCow attribute for the existing files.
> 
> Should we currently use a separate partition for VMs and mount it with
> nodatacow option to avoid that issue?

So, Linux for a while now has the FALLOC_FL_UNSHARE_RANGE flag to
fallocate to unshare previously shared extents.  It still lacks an
implementation for btrfs, but it seems to be the interface that you
want.

Re: Will Btrfs have an official command to "uncow" existing files?

[Zygo Blaxell]

On Sun, Jan 22, 2023 at 09:27:33PM +0100, Goffredo Baroncelli wrote:
> On 22/01/2023 12.41, Cerem Cem ASLAN wrote:
> > Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html
> > 
> > The problem with the "chattr +C ..., move back and forth" approach is
> > that the VM folder is about 300GB and I have ~100GB of free space,
> > plus,
> 
> This can be solvable: it should be possible to make a tool that for
> each unit of copy (eg each 1 GB) does:
> - copy the data from the COW file to the NOCOW file
> - remove the data copied from the NOCOW file (using fallocate+FALLOC_FL_PUNCH_HOLE)
> 
> So you can avoid to have two FULL copy of the same file (in pseudo code:
> 
> block_size = 1024*1024*1024;
> while (pos < file_len) {
> 	l = min(block_size, file_len - pos);
> 	len_copied = copy(srcfd, dstfd, pos, l);
> 	fallocate(srcfd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, pos, l);
> 
> 	pos += l;
> }
> 
> 
> end pseudo code)
> 
> I don't know if there is an algorithm that prevent the data loss in case of
> an interruption of the copy. May be that it exists... We need a file where
> we could log the status.

Start at the end of the file and work backwards, making the source
file shorter with truncate.  The size of the original file provides a
built-in checkpoint that you can resume from, at worst, copying the last
1GB of the file again after resuming.  There's no danger of accidentally
copying a bunch of zeros that would be left behind by PUNCH_HOLE.

Don't forget to open the dst file without O_TRUNC, and call fsync()
(or ideally sync() to force a full commit and avoid possible log tree
replay issues) between writing the copy and truncating the original.

Note that if you're doing this with 300GB of data on a filesystem with
100GB free, and the files are already fragmented because they were used
with a VM, then you'll end up with massively fragmented nodatacow files,
because the parts that are deleted by truncate() will leave behind tiny
free space holes, and those holes will be the only space available after
the first 25% of the copy.  You can partially mitigate this by stopping
to run balance every now and then to defragment free space during the
copy, but at that point you'll be moving 1200+ GB of data around, not
counting metadata tree updates.

Honestly, attaching a second 400GB device and copying the files
straight from one to the other and back will be a *better solution*
to this problem.

> > I have multiple copies which will require that 300GB to
> > re-transfer after deleting all previous snapshots (because there is no
> > enough free space on those backup hard disks).

Note that snapshots are not compatible with nodatacow.  nodatacow will
be disabled for each extent where another reference exists.

> This is a more stronger requirement; but unfortunately if you copy the data you
> will end to have two copy of the data which before was shared between the snapshots.

If there are multiple snapshots of the file, you'll have to do this
in parallel with all of them, cloning into each nodatacow file on
the snapshots from the newly copied nocow file after each GB copied.

Of course all of these files would really be datacow despite the +C
attribute, since they have shared extents.

> > So, we really need to set the NoCow attribute for the existing files.

The main reason why it's not allowed is because nodatacow implies
nodatasum.  datacow can be enabled and disabled for each extent (e.g. when
a snapshot of a nodatacow file is made, the file becomes implicitly
datacow) because nodatacow doesn't impose additional tree lookup costs
compared to datacow.  datasum stores data in a separate tree, and there
are IO costs which must be paid for datasum whether the csum exists in
the tree or not, so btrfs needs to know whether there's a csum _before_
doing a lookup in the csum tree.

Currently the nodatasum flag is stored in the btrfs inode.  Every inode
that references a extent has a separate nodatasum flag, and all of those
inodes must agree on the same flag value.  It is not permitted to change
the nodatasum attribute on a file while it has extents, specifically
to prevent issues with data extents having csums when accessed through
some files but not having csums when accessed through other files.
It's also not allowed to make reflink copies of data from a datasum file
to a non-datasum file (or vice versa) for the same reason:  every inode
referencing an extent must have the same datasum flag value.

Consider what happens when an extent with nodatasum is deleted, but
references previously existed to the extent with datasum--the csums from
the deleted extent wouldn't be removed, causing an EEXIST panic later
when new data is written to the freed blocks, but csums already exist
for those blocks in the tree.  That would be bad, so it's not allowed.

This could be fixed in btrfs by stealing one of the bits in a file extent
item, then using it to indicate whether the data extent has csums present.
This would make datasums behave the same way compression does--each extent
decides whether it is compressed or not, so you can freely enable or
disable compression on a file, and it only affects what happens to new
extents written in the file.  When an extent is reflinked, the reflink
has the same compression setting as the original extent.  btrfs could do
that with datasums too, and then a file could be freely changed from
datacow+datasum to nodatacow+nodatasum, which would only affect the
behavior of extents written to the file in the future.  The csum tree
lookup costs are avoided because btrfs knows whether a csum exists from
the file extent item.

With this new on-disk structure, it would be possible to iterate over all
the extents in a file, change their file extent items to be nodatasum,
then delete the csums on the extent when there are no references remaining
to them, all without copying any data.  That wouldn't be worse than
running balance in terms of metadata IO costs (data costs are zero).

> > Should we currently use a separate partition for VMs and mount it with
> > nodatacow option to avoid that issue?
> 
> Not really, it is enough to do a chmod -C on the directory where
> the VM images are stored.
> 
> -- 
> gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
> Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
> 

Re: Will Btrfs have an official command to "uncow" existing files?

[Goffredo Baroncelli]

On 22/01/2023 12.41, Cerem Cem ASLAN wrote:
> Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html
> 
> The problem with the "chattr +C ..., move back and forth" approach is
> that the VM folder is about 300GB and I have ~100GB of free space,
> plus, 

This can be solvable: it should be possible to make a tool that for
each unit of copy (eg each 1 GB) does:
- copy the data from the COW file to the NOCOW file
- remove the data copied from the NOCOW file (using fallocate+FALLOC_FL_PUNCH_HOLE)

So you can avoid to have two FULL copy of the same file (in pseudo code:

block_size = 1024*1024*1024;
while (pos < file_len) {
	l = min(block_size, file_len - pos);
	len_copied = copy(srcfd, dstfd, pos, l);
	fallocate(srcfd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, pos, l);

	pos += l;
}


end pseudo code)

I don't know if there is an algorithm that prevent the data loss in case of
an interruption of the copy. May be that it exists... We need a file where
we could log the status.

> I have multiple copies which will require that 300GB to
> re-transfer after deleting all previous snapshots (because there is no
> enough free space on those backup hard disks).
  
This is a more stronger requirement; but unfortunately if you copy the data you
will end to have two copy of the data which before was shared between the snapshots.


> So, we really need to set the NoCow attribute for the existing files.
> 
> Should we currently use a separate partition for VMs and mount it with
> nodatacow option to avoid that issue?

Not really, it is enough to do a chmod -C on the directory where
the VM images are stored.

-- 
gpg @keyserver.linux.it: Goffredo Baroncelli <kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5

Re: Will Btrfs have an official command to "uncow" existing files?

[Forza]

---- From: Cerem Cem ASLAN <ceremcem@ceremcem.net> -- Sent: 2023-01-22 - 12:41 ----

> Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html
> 
> The problem with the "chattr +C ..., move back and forth" approach is
> that the VM folder is about 300GB and I have ~100GB of free space,
> plus, I have multiple copies which will require that 300GB to
> re-transfer after deleting all previous snapshots (because there is no
> enough free space on those backup hard disks).
> 
> So, we really need to set the NoCow attribute for the existing files.
> 
> Should we currently use a separate partition for VMs and mount it with
> nodatacow option to avoid that issue?


The issue with nocow, is that it also disables data checksums. This is the reason why btrfs won't allow conversion between nocow and cow, as there is no way to determine if there is data corruption. 

This brings me to the second question. Do you use a filesystem that does data verification (such as btrfs) inside the guest VM? If not, then how do you guarantee the integrity? 

If you do not need data integrity, then it might be useful to use LVM with thin volumes instead of btrfs as your backing storage. 

Will Btrfs have an official command to "uncow" existing files?

[Cerem Cem ASLAN]

Original post is here: https://www.spinics.net/lists/linux-btrfs/msg58055.html

The problem with the "chattr +C ..., move back and forth" approach is
that the VM folder is about 300GB and I have ~100GB of free space,
plus, I have multiple copies which will require that 300GB to
re-transfer after deleting all previous snapshots (because there is no
enough free space on those backup hard disks).

So, we really need to set the NoCow attribute for the existing files.

Should we currently use a separate partition for VMs and mount it with
nodatacow option to avoid that issue?