linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* btrfs send / receive via netcat, fails halfway?
@ 2021-01-10 10:34  
  2021-01-10 11:07 ` Forza
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From:   @ 2021-01-10 10:34 UTC (permalink / raw)
  To: linux-btrfs

­I'm trying to transfer a btrfs snapshot via the network.

First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.

btrfs subvolume delete /mnt/rec/snapshots/*
receive side:
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0

sending side:
# btrfs send  /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open



Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete. 
receive side:
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 1
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 2
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 3
read(net): Connection reset by peer
ERROR: short read from stream: expected 49183 read 10450
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 4
ERROR: cannot find parent subvolume
write(stdout): Broken pipe

sending side:
# btrfs send  /mnt/send/snapshots/0 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/1 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/1
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/1 /mnt/send/snapshots/2 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/2
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/2 /mnt/send/snapshots/3 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/3
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/3 /mnt/send/snapshots/4 | nc -v -c  127.0.0.1 6790
At subvol /mnt/send/snapshots/4
localhost [127.0.0.1] 6790 (hnmp) open
write(net): Connection reset by peer




---

Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!



^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: btrfs send / receive via netcat, fails halfway?
  2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?  
@ 2021-01-10 11:07 ` Forza
  2021-01-10 11:37 ` Roman Mamedov
  2021-01-10 11:47 ` Hugo Mills
  2 siblings, 0 replies; 6+ messages in thread
From: Forza @ 2021-01-10 11:07 UTC (permalink / raw)
  To: Cedric.dewijs, linux-btrfs



On 2021-01-10 11:34, Cedric.dewijs@eclipso.eu wrote:
> ­I'm trying to transfer a btrfs snapshot via the network.
> 
> First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.
> 
> btrfs subvolume delete /mnt/rec/snapshots/*
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
> 
> sending side:
> # btrfs send  /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
> At subvol /mnt/send/snapshots/0
> localhost [127.0.0.1] 6790 (hnmp) open
> 
> 
> 
> Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete.
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 1
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 2
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 3
> read(net): Connection reset by peer
> ERROR: short read from stream: expected 49183 read 10450
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 4
> ERROR: cannot find parent subvolume
> write(stdout): Broken pipe
> 
> sending side:
> # btrfs send  /mnt/send/snapshots/0 | nc -v -c 127.0.0.1 6790
> At subvol /mnt/send/snapshots/0
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/1 | nc -v -c  127.0.0.1 6790
> At subvol /mnt/send/snapshots/1
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/1 /mnt/send/snapshots/2 | nc -v -c  127.0.0.1 6790
> At subvol /mnt/send/snapshots/2
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/2 /mnt/send/snapshots/3 | nc -v -c  127.0.0.1 6790
> At subvol /mnt/send/snapshots/3
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/3 /mnt/send/snapshots/4 | nc -v -c  127.0.0.1 6790
> At subvol /mnt/send/snapshots/4
> localhost [127.0.0.1] 6790 (hnmp) open
> write(net): Connection reset by peer
> 
> 
> 

Hi,

Haven't used netcat for btrfs send, so I am not sure why you have these 
issues. Have you tried mbuffer instead? This is what I use when I don't 
need to go over ssh.

Mbuffer[1] allows you to specify remote host and port. It also has an 
async buffer so it should be a bit faster too. In fact, it is useful on 
a localhost too, for example when sending snaps to a slower local backup 
media.

Receiving side:
# mbuffer -I port | btrfs receive /mnt/rec/snapshots

Sending side:
# btrfs send /mnt/send/snapshots/0 | mbuffer -O remotehost:port

Good Luck!

//Forza

[1]https://www.maier-komor.de/mbuffer.html






^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: btrfs send / receive via netcat, fails halfway?
  2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?  
  2021-01-10 11:07 ` Forza
@ 2021-01-10 11:37 ` Roman Mamedov
  2021-01-10 12:38   `  
  2021-01-10 11:47 ` Hugo Mills
  2 siblings, 1 reply; 6+ messages in thread
From: Roman Mamedov @ 2021-01-10 11:37 UTC (permalink / raw)
  To:  ; +Cc: linux-btrfs

On Sun, 10 Jan 2021 11:34:27 +0100
" " <Cedric.dewijs@eclipso.eu> wrote:

> ­I'm trying to transfer a btrfs snapshot via the network.
> 
> First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.

It is a common annoyance that NC doesn't exit in such scenario and needs to be
Ctrl-C'ed after verifying that the transfer is over.

Instead, at host2 try:

  ssh host1 "btrfs send ..." | btrfs receive ...

Also much more secure.

-- 
With respect,
Roman

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: btrfs send / receive via netcat, fails halfway?
  2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?  
  2021-01-10 11:07 ` Forza
  2021-01-10 11:37 ` Roman Mamedov
@ 2021-01-10 11:47 ` Hugo Mills
  2021-01-10 13:27   ` Hugo Mills
  2 siblings, 1 reply; 6+ messages in thread
From: Hugo Mills @ 2021-01-10 11:47 UTC (permalink / raw)
  To: Cedric.dewijs; +Cc: linux-btrfs

On Sun, Jan 10, 2021 at 11:34:27AM +0100,   wrote:
> ­I'm trying to transfer a btrfs snapshot via the network.
> 
> First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.
> 
> btrfs subvolume delete /mnt/rec/snapshots/*
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
> 
> sending side:
> # btrfs send  /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
> At subvol /mnt/send/snapshots/0
> localhost [127.0.0.1] 6790 (hnmp) open

   Use -q 15 on the sending side. That will exit after 15 seconds of
no activity from the send process.

> Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete. 
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 1
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 2
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 3
> read(net): Connection reset by peer
> ERROR: short read from stream: expected 49183 read 10450

   This failed because of a network disconnect.

> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 4
> ERROR: cannot find parent subvolume
> write(stdout): Broken pipe

   This is expected because the previous one failed.

   There's no btrfs problem here. You just need better error handling
(to retry a failed transfer, for example).

   Hugo.

-- 
Hugo Mills             | How do you become King? You stand in the marketplace
hugo@... carfax.org.uk | and announce you're going to tax everyone. If you
http://carfax.org.uk/  | get out alive, you're King.
PGP: E2AB1DE4          |                                        Harry Harrison

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: Re: btrfs send / receive via netcat, fails halfway?
  2021-01-10 11:37 ` Roman Mamedov
@ 2021-01-10 12:38   `  
  0 siblings, 0 replies; 6+ messages in thread
From:   @ 2021-01-10 12:38 UTC (permalink / raw)
  To: Roman Mamedov; +Cc: linux-btrfs



It is a common annoyance that NC doesn't exit in such scenario and needs
to be
Ctrl-C'ed after verifying that the transfer is over.

Instead, at host2 try:

  ssh host1 "btrfs send ..." | btrfs receive ...

Also much more secure.

-- 
With respect,
Roman

I was practicing on a test system, the real transfer was supposed to be between two systems on a LAN. I didn't want encryption, as my system then only is capable of sending 30MB/sec, instead of the 110MB/sec over the 1GB network.


---

Take your mailboxes with you. Free, fast and secure Mail &amp; Cloud: https://www.eclipso.eu - Time to change!



^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: btrfs send / receive via netcat, fails halfway?
  2021-01-10 11:47 ` Hugo Mills
@ 2021-01-10 13:27   ` Hugo Mills
  0 siblings, 0 replies; 6+ messages in thread
From: Hugo Mills @ 2021-01-10 13:27 UTC (permalink / raw)
  To: Cedric.dewijs, linux-btrfs

   By the way, Cedric, your SMTP server is rejecting mail from mine
with a "forged HELO" error. I'm not sure why, but I've not knowingly
encountered this with anyone else's mail server.

   Hugo.

-- 
Hugo Mills             | The last man on Earth sat in a room. Suddenly, there
hugo@... carfax.org.uk | was a knock at the door.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                        Frederic Brown

^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-10 13:28 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?  
2021-01-10 11:07 ` Forza
2021-01-10 11:37 ` Roman Mamedov
2021-01-10 12:38   `  
2021-01-10 11:47 ` Hugo Mills
2021-01-10 13:27   ` Hugo Mills

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).