Re: Incremental receive completes succesfully despite missing files

[Dennis Katsonis <dennisk@netspace.net.au>]

On 1/25/19 4:22 AM, Chris Murphy wrote:
> On Thu, Jan 24, 2019 at 3:40 AM Dennis K <dennisk@netspace.net.au> wrote:
>>
>> The fact is, this thread is the first time I've seen explicitly written
>> that parents must be the same at receiving and sending ends, or else
>> btrfs-send/receive will produce a subvolume which differs from the source.
> 
> The central user error, as well as btrfs-progs bug is the failure to
> meet the requirement that the source(s) be snapshots. Either a full
> send, or an incremental send, whether with -p or -c, all of them must
> be snapshots. And none of yours were snapshots. They were read-only
> subvolumes made using 'btrfs property' to set the read-only flag, and
> those are not snapshots.

Doesn't matter even if you only send snapshots and never set a subvolume
to ro to please btrfs-send.

# btrfs sub create 1
# touch 1/file1
# btrfs sub snap -r 1 2
# btrfs send 2 | btrfs receive /destination
!# sudo btrfs prop set /destination/2/ ro false
!# rm /destination/2/file1
# touch 1/file2
# sudo btrfs sub snap -r 1 3
# btrfs send -p 2 3 | btrfs receive /storage2/
# ls /storage2/3/
file2
# ls 3/
file1 file2

The lines with the '!' are the source of the trouble, but it doesn't
have to be rm.  You could modify file 1 instead.  It is obvious that
users shouldn't do that, AFTER you've run through this sequence.

Note that snapshot 2 at the receiving end was never set back to ro.


> 
> The btrfs send command should have explicitly failed on that alone.
> 
> In my reproducer of a related user error and bug, both were snapshots,
> but they were unrelated snapshots, therefore incremental send was
> impossible, and the only explanation for such a command is the user is
> confused, made a mistake, and therefore the command should have
> failed.
> 
>>
>> A user could from the descriptions of how send/receive formats its data
>> strea, and from interpreting "clone sources" in a manner contrary to how
>> its used in the man page and the mention of incrementals, conclude that
>> send/receive is sensitive to changes at the receiving side, but its far,
>> far more likely that they will start using feature like send/receive and
>> setting snapshots to ro before they delve into how it works its magic.
> 
> Uhhh I think this is a specious statement if you think users are
> likely to delve into 'btrfs property' and somehow think setting a
> subvolume ro is making a snapshot. You're the first I've encountered
> who arrived at this logic. For now almost a decade people arrive at
> 'btrfs subvolume snapshot' as the way to make snapshots, and the -r
> flag to make them read-only snapshots.




> https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Snapshots
> 
> That page only describes snapshots in terms of the 'btrfs subvolume
> snapshot' command - the 'btrfs property' command isn't found anywhere
> on that page. So I don't actually know how you came to think setting
> the ro property on a subvolume to true, makes the subvolume a
> snapshot. I think you did it through assumption. And while there might
> be a way to fix some things to avoid such confusion in the future, I
> do think it's an edge case.
> 
> 
> The man 8 btrfs man page could amend:
> 
>        subvolume
>            Create/delete/list/manage btrfs subvolume.
> 
> to
> 
>        subvolume
>            Create/delete/list/manage btrfs subvolumes and snapshots.
> 
> it is something of btrfs geekdom that snapshots are subvolumes; but
> still not all subvolumes are snapshots. So...yeah, clarity is a good
> thing even if there some redundancy.
> 
>> Anyway, rsync sends incremental change, and its ability to replicate a
>> directory tree isn't predicated on the files present at the receiving
>> end being the same as the last time it was run...
> 
> What? Yes it is. By default it compares size and time stamp on both
> sides. If you use -c it uses checksums to compare. From the rsync man
> page:

# touch file1
# touch file2
# mkdir /destination/snap1
# rsync -r . /destination/snap1
# rm /destination/snap1/file1
# touch file3
# mkdir /destination/snap2
# cp -r /destination/snap1 /destination/snap2
# rsync -r . /destination/snap2
# ls /destination/snap2
file1 file2 file3
# ls .
file1 file2 file3

You can also add an addition "rm /destination/snap2/file1" or even "rm
-rf /destination/snap2/*" before the last rsync, and it still gives you
a full replication.

The expectation after running rsync without any explicit additional
inclusion/exclusion clauses is replication, which is what it provides.
Having deleted files at the destination end in between doesn't cause
rsync to fail meeting that expectation.



> 
>> Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in  size  or  in  last-modified
>        time.
> 
> Rsync does not keep a database on source or destination. It absolutely
> is comparing both sides, and if they aren't the same, the file is
> synced from the source to the destination.

> 
> Also, the man page for btrfs receive very clearly says the receive
> side snapshot must be present and must not have been changed since the
> last receive, or it will fail.
> 
>>       btrfs receive will fail in the following cases:
>>       1. receiving subvolume already exists
>>       2. previously received subvolume has been changed after it was received
> 
> I'm still really not following where your confusion stems from, and
> therefore I'm not sure what needs fixing other than the items I've
> already mentioned - which itself at least would have stopped you in
> your tracks, to go dig deeper or ask questions before arriving at the
> understandably confusing results you were getting.
> 

From the man page

"       btrfs receive will fail in the following cases:

        1. receiving subvolume already exists

        2. previously received subvolume has been changed after it was
received

        3. default subvolume has changed or you didn’t mount the
filesystem at the toplevel subvolume

       A subvolume is made read-only after the receiving process
finishes successfully (see BUGS below).
"

Point 2 is what you are referring to, but it doesn't fail.  It reports
success and doesn't return an error code to the shell.  So the
documentation could be improved by reflecting this fact, which can avoid
users assuming that the snapshots at the receiving end must be the same
because it didn't fail.

Also, the btrfs-send man page could be improved, by including parents as
well as clone sources as being required to be the same on both ends.

That is all.