synchronize a btrfs snapshot via network with resume support?

synchronize a btrfs snapshot via network with resume support?

­I have 2 PC's, one called master, and one called slave. On the master, I have a btrfs filesystem with real-only snapshots that are made daily. On the slave PC, I would like to create a btrfs filesystem with the exact same contents as the master. The two PC's are connected via internet, so interruptions in the connection are to be expected.

Btrfs send can convert the differences between 2 read only snapshots into a stream of instructions. This stream can then be send via ssh to a remote computer, and fed into btrfs receive, so that the snapshot is recreated at the remote computer.

The problem with this approach is that the transmission of the entire delta between two snapshots has to succeed in one go. If there's an interruption in the connection, the complete transmission has to be restarted. This is a problem if the delta between two snapshots is large, or if the connection is slow.

I'm looking for a program that can synchronize a btrfs snapshot via a network, and supports resuming of interrupted transfers.

Buttersink [1] claims it can do Resumable, checksummed multi-part uploads to S3 as a back-end, but not between two PC's via ssh.

Does anybody know a program that is capable of doing this?

---

Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!

Re: synchronize a btrfs snapshot via network with resume support?

[Graham Cobb]

On 01/01/2021 14:42, Cedric.dewijs@eclipso.eu wrote:
...
> I'm looking for a program that can synchronize a btrfs snapshot via a network, and supports resuming of interrupted transfers.

Not an answer to your question... the way I would solve your problem is
to do the "btrfs send" to a local file on master, reliably transfer the
file, then do the "btrfs receive" on slave from the file. Reliable file
transfer programs exist (you can even just use rsync --inplace if I
remember correctly).

Unfortunately, that requires that you have enough space on both ends to
store the btrfs-send file.

> 
> Buttersink [1] claims it can do Resumable, checksummed multi-part uploads to S3 as a back-end, but not between two PC's via ssh.

I don't use Buttersink. But if it can do that without storing the file
locally on either end (which I would be slightly surprised about) then
it sounds like that might be the way to do it: essentially you would be
paying AWS for the temporary filespace needed.

Graham