* received uuid not set btrfs send/receive
@ 2021-01-17 18:49 Anders Halman
2021-01-17 19:21 ` Hugo Mills
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Anders Halman @ 2021-01-17 18:49 UTC (permalink / raw)
To: linux-btrfs
Hello,
I try to backup my laptop over an unreliable slow internet connection to
a even slower Raspberry Pi.
To bootstrap the backup I used the following:
# local
btrfs send root.send.ro | pigz | split --verbose -d -b 1G
rsync -aHAXxv --numeric-ids --partial --progress -e "ssh -T -o
Compression=no -x" x* remote-host:/mnt/backup/btrfs-backup/
# remote
cat x* > split.gz
pigz -d split.gz
btrfs receive -f split
worked nicely. But I don't understand why the "received uuid" on the
remote site in blank.
I tried it locally with smaller volumes and it worked.
The 'split' file contains the correct uuid, but it is not set (remote).
remote$ btrfs receive --dump -f split | head
subvol ./root.send.ro uuid=99a34963-3506-7e4c-a82d-93e337191684
transid=1232187
local$ sudo btrfs sub show root.send.ro| grep -i uuid:
UUID: 99a34963-3506-7e4c-a82d-93e337191684
Questions:
- Is there a way to set the "received uuid"?
- Is it a matter of btrfs-progs version difference?
- What whould be a better approach?
Thank you
----
# local
root@fos ~$ uname -a
Linux fos 5.9.16-200.fc33.x86_64 #1 SMP Mon Dec 21 14:08:22 UTC 2020
x86_64 x86_64 x86_64 GNU/Linux
root@fos ~$ btrfs --version
btrfs-progs v5.9
root@fos ~$ btrfs fi show
Label: 'DATA' uuid: b6e675b3-84e3-4869-b858-218c5f0ac5ad
Total devices 1 FS bytes used 402.17GiB
devid 1 size 464.27GiB used 414.06GiB path
/dev/mapper/luks-e4e69cfa-faae-4af8-93f5-7b21b25ab4e6
root@fos ~$ btrfs fi df /btrfs-root/
Data, single: total=404.00GiB, used=397.80GiB
System, DUP: total=32.00MiB, used=64.00KiB
Metadata, DUP: total=5.00GiB, used=4.38GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
# remote
root@pih:~# uname -a
Linux pih 5.4.72+ #1356 Thu Oct 22 13:56:00 BST 2020 armv6l GNU/Linux
root@pih:~# btrfs --version
btrfs-progs v4.20.1
root@pih:~# btrfs fi show
Label: 'DATA' uuid: 6be1e09c-d1a5-469d-932b-a8d1c339afae
Total devices 1 FS bytes used 377.57GiB
devid 2 size 931.51GiB used 383.06GiB path
/dev/mapper/luks_open_backup0
root@pih:~# btrfs fi df /mnt/backup
Data, single: total=375.00GiB, used=374.25GiB
System, DUP: total=32.00MiB, used=64.00KiB
Metadata, DUP: total=4.00GiB, used=3.32GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
dmesg is empty for the time of import/btrfs receive.
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: received uuid not set btrfs send/receive
2021-01-17 18:49 received uuid not set btrfs send/receive Anders Halman
@ 2021-01-17 19:21 ` Hugo Mills
2021-01-17 21:07 ` Chris Murphy
2021-01-18 5:49 ` Andrei Borzenkov
2 siblings, 0 replies; 6+ messages in thread
From: Hugo Mills @ 2021-01-17 19:21 UTC (permalink / raw)
To: Anders Halman; +Cc: linux-btrfs
On Sun, Jan 17, 2021 at 10:49:26AM -0800, Anders Halman wrote:
> Hello,
>
> I try to backup my laptop over an unreliable slow internet connection to a
> even slower Raspberry Pi.
>
> To bootstrap the backup I used the following:
>
> # local
> btrfs send root.send.ro | pigz | split --verbose -d -b 1G
> rsync -aHAXxv --numeric-ids --partial --progress -e "ssh -T -o
> Compression=no -x" x* remote-host:/mnt/backup/btrfs-backup/
>
> # remote
> cat x* > split.gz
> pigz -d split.gz
> btrfs receive -f split
> worked nicely. But I don't understand why the "received uuid" on the remote
> site in blank.
Are you doing the receive as root?
Hugo.
> I tried it locally with smaller volumes and it worked.
>
> The 'split' file contains the correct uuid, but it is not set (remote).
>
> remote$ btrfs receive --dump -f split | head
> subvol ./root.send.ro uuid=99a34963-3506-7e4c-a82d-93e337191684
> transid=1232187
>
> local$ sudo btrfs sub show root.send.ro| grep -i uuid:
> UUID: 99a34963-3506-7e4c-a82d-93e337191684
>
>
> Questions:
>
> - Is there a way to set the "received uuid"?
> - Is it a matter of btrfs-progs version difference?
> - What whould be a better approach?
>
>
> Thank you
>
>
> ----
>
> # local
>
> root@fos ~$ uname -a
> Linux fos 5.9.16-200.fc33.x86_64 #1 SMP Mon Dec 21 14:08:22 UTC 2020 x86_64
> x86_64 x86_64 GNU/Linux
>
> root@fos ~$ btrfs --version
> btrfs-progs v5.9
>
> root@fos ~$ btrfs fi show
> Label: 'DATA' uuid: b6e675b3-84e3-4869-b858-218c5f0ac5ad
> Total devices 1 FS bytes used 402.17GiB
> devid 1 size 464.27GiB used 414.06GiB path
> /dev/mapper/luks-e4e69cfa-faae-4af8-93f5-7b21b25ab4e6
>
> root@fos ~$ btrfs fi df /btrfs-root/
> Data, single: total=404.00GiB, used=397.80GiB
> System, DUP: total=32.00MiB, used=64.00KiB
> Metadata, DUP: total=5.00GiB, used=4.38GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
> # remote
> root@pih:~# uname -a
> Linux pih 5.4.72+ #1356 Thu Oct 22 13:56:00 BST 2020 armv6l GNU/Linux
>
> root@pih:~# btrfs --version
> btrfs-progs v4.20.1
>
> root@pih:~# btrfs fi show
> Label: 'DATA' uuid: 6be1e09c-d1a5-469d-932b-a8d1c339afae
> Total devices 1 FS bytes used 377.57GiB
> devid 2 size 931.51GiB used 383.06GiB path
> /dev/mapper/luks_open_backup0
>
> root@pih:~# btrfs fi df /mnt/backup
> Data, single: total=375.00GiB, used=374.25GiB
> System, DUP: total=32.00MiB, used=64.00KiB
> Metadata, DUP: total=4.00GiB, used=3.32GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
> dmesg is empty for the time of import/btrfs receive.
--
Hugo Mills | If it ain't broke, hit it again.
hugo@... carfax.org.uk |
http://carfax.org.uk/ |
PGP: E2AB1DE4 | Foon
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: received uuid not set btrfs send/receive
2021-01-17 18:49 received uuid not set btrfs send/receive Anders Halman
2021-01-17 19:21 ` Hugo Mills
@ 2021-01-17 21:07 ` Chris Murphy
2021-01-20 3:19 ` Anders Halman
2021-01-21 3:44 ` [SOLVED] " Anders Halman
2021-01-18 5:49 ` Andrei Borzenkov
2 siblings, 2 replies; 6+ messages in thread
From: Chris Murphy @ 2021-01-17 21:07 UTC (permalink / raw)
To: Anders Halman; +Cc: Btrfs BTRFS
On Sun, Jan 17, 2021 at 11:51 AM Anders Halman <anders.halman@gmail.com> wrote:
>
> Hello,
>
> I try to backup my laptop over an unreliable slow internet connection to
> a even slower Raspberry Pi.
>
> To bootstrap the backup I used the following:
>
> # local
> btrfs send root.send.ro | pigz | split --verbose -d -b 1G
> rsync -aHAXxv --numeric-ids --partial --progress -e "ssh -T -o
> Compression=no -x" x* remote-host:/mnt/backup/btrfs-backup/
>
> # remote
> cat x* > split.gz
> pigz -d split.gz
> btrfs receive -f split
>
> worked nicely. But I don't understand why the "received uuid" on the
> remote site in blank.
> I tried it locally with smaller volumes and it worked.
I suggest using -v or -vv on the receive side to dig into why the
receive is failing. Setting the received uuid is one of the last
things performed on receive, so if it's not set it suggests the
receive isn't finished.
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: received uuid not set btrfs send/receive
2021-01-17 21:07 ` Chris Murphy
@ 2021-01-20 3:19 ` Anders Halman
2021-01-21 3:44 ` [SOLVED] " Anders Halman
1 sibling, 0 replies; 6+ messages in thread
From: Anders Halman @ 2021-01-20 3:19 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
Sorry for answering so late. I compiled a more recent (v5.9) version of
btrfs-progs first.
The unpacking/receiving took about 2 days on the raspberry.
$ nohup btrfs receive -v -f splitter . &
$ tail -n1 nohup.out
ERROR: short read from stream: expected 49233 read 44997
thanks to your -v suggestions I got a little bit more information.
first I figured the files on the local and remote host are not the same,
by md5sum.
So I will solve this issue first. Is there any recommended way to
transfer btrfs subvolumes over an unstable connection?
Am 17.01.21 um 13:07 schrieb Chris Murphy:
> On Sun, Jan 17, 2021 at 11:51 AM Anders Halman <anders.halman@gmail.com> wrote:
>> Hello,
>>
>> I try to backup my laptop over an unreliable slow internet connection to
>> a even slower Raspberry Pi.
>>
>> To bootstrap the backup I used the following:
>>
>> # local
>> btrfs send root.send.ro | pigz | split --verbose -d -b 1G
>> rsync -aHAXxv --numeric-ids --partial --progress -e "ssh -T -o
>> Compression=no -x" x* remote-host:/mnt/backup/btrfs-backup/
>>
>> # remote
>> cat x* > split.gz
>> pigz -d split.gz
>> btrfs receive -f split
>>
>> worked nicely. But I don't understand why the "received uuid" on the
>> remote site in blank.
>> I tried it locally with smaller volumes and it worked.
> I suggest using -v or -vv on the receive side to dig into why the
> receive is failing. Setting the received uuid is one of the last
> things performed on receive, so if it's not set it suggests the
> receive isn't finished.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [SOLVED] received uuid not set btrfs send/receive
2021-01-17 21:07 ` Chris Murphy
2021-01-20 3:19 ` Anders Halman
@ 2021-01-21 3:44 ` Anders Halman
1 sibling, 0 replies; 6+ messages in thread
From: Anders Halman @ 2021-01-21 3:44 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS
The problem was a corrupt "btrfs send file" or to be more specific, the
file got corrupt somewhere on the line of transport
To recap the problem for further reference:
check the import (btrfs receive) with the -v option like so:
$ nohup btrfs receive -v -f root.receive.file . &
the import is successful, thus the received uuid is set when you get
something like ....
$ tail -n1 nohup.out
BTRFS_IOC_SET_RECEIVED_SUBVOL uuid=99a34963-3506-7e4c-a82d-93e337191684,
stransid=1232187
... after "btrfs receive" is done.
make sure to double check the file size and checksum (md5sum) of "btrfs
send file" and "btrfs receive file".
I don't know exactly where the corruption happened, but in the second
attempt I successful combined the import like so:
$ cat x* | pigz -d > root.receive.file
Thanks for the support
Am 17.01.21 um 13:07 schrieb Chris Murphy:
> On Sun, Jan 17, 2021 at 11:51 AM Anders Halman <anders.halman@gmail.com> wrote:
>> Hello,
>>
>> I try to backup my laptop over an unreliable slow internet connection to
>> a even slower Raspberry Pi.
>>
>> To bootstrap the backup I used the following:
>>
>> # local
>> btrfs send root.send.ro | pigz | split --verbose -d -b 1G
>> rsync -aHAXxv --numeric-ids --partial --progress -e "ssh -T -o
>> Compression=no -x" x* remote-host:/mnt/backup/btrfs-backup/
>>
>> # remote
>> cat x* > split.gz
>> pigz -d split.gz
>> btrfs receive -f split
>>
>> worked nicely. But I don't understand why the "received uuid" on the
>> remote site in blank.
>> I tried it locally with smaller volumes and it worked.
> I suggest using -v or -vv on the receive side to dig into why the
> receive is failing. Setting the received uuid is one of the last
> things performed on receive, so if it's not set it suggests the
> receive isn't finished.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: received uuid not set btrfs send/receive
2021-01-17 18:49 received uuid not set btrfs send/receive Anders Halman
2021-01-17 19:21 ` Hugo Mills
2021-01-17 21:07 ` Chris Murphy
@ 2021-01-18 5:49 ` Andrei Borzenkov
2 siblings, 0 replies; 6+ messages in thread
From: Andrei Borzenkov @ 2021-01-18 5:49 UTC (permalink / raw)
To: Anders Halman, linux-btrfs
17.01.2021 21:49, Anders Halman пишет:
> Hello,
>
> I try to backup my laptop over an unreliable slow internet connection to
> a even slower Raspberry Pi.
>
> To bootstrap the backup I used the following:
>
> # local
> btrfs send root.send.ro | pigz | split --verbose -d -b 1G
> rsync -aHAXxv --numeric-ids --partial --progress -e "ssh -T -o
> Compression=no -x" x* remote-host:/mnt/backup/btrfs-backup/
>
> # remote
> cat x* > split.gz
> pigz -d split.gz
> btrfs receive -f split
>
This command is missing argument.
> worked nicely.
It is impossible, you did not tell "btrfs receive" where to store
incoming data.
bor@bor-Latitude-E5450:~/src/btrfs-progs$ btrfs receive
btrfs receive: exactly 1 argument expected, 0 given
> But I don't understand why the "received uuid" on the
> remote site in blank.
Where is it blank? Show commands you used to check it.
> I tried it locally with smaller volumes and it worked.
>
> The 'split' file contains the correct uuid, but it is not set (remote).
>
> remote$ btrfs receive --dump -f split | head
> subvol ./root.send.ro uuid=99a34963-3506-7e4c-a82d-93e337191684
> transid=1232187
>
> local$ sudo btrfs sub show root.send.ro| grep -i uuid:
> UUID: 99a34963-3506-7e4c-a82d-93e337191684
>
>
> Questions:
>
> - Is there a way to set the "received uuid"?
> - Is it a matter of btrfs-progs version difference?
> - What whould be a better approach?
>
>
> Thank you
>
>
> ----
>
> # local
>
> root@fos ~$ uname -a
> Linux fos 5.9.16-200.fc33.x86_64 #1 SMP Mon Dec 21 14:08:22 UTC 2020
> x86_64 x86_64 x86_64 GNU/Linux
>
> root@fos ~$ btrfs --version
> btrfs-progs v5.9
>
> root@fos ~$ btrfs fi show
> Label: 'DATA' uuid: b6e675b3-84e3-4869-b858-218c5f0ac5ad
> Total devices 1 FS bytes used 402.17GiB
> devid 1 size 464.27GiB used 414.06GiB path
> /dev/mapper/luks-e4e69cfa-faae-4af8-93f5-7b21b25ab4e6
>
> root@fos ~$ btrfs fi df /btrfs-root/
> Data, single: total=404.00GiB, used=397.80GiB
> System, DUP: total=32.00MiB, used=64.00KiB
> Metadata, DUP: total=5.00GiB, used=4.38GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
> # remote
> root@pih:~# uname -a
> Linux pih 5.4.72+ #1356 Thu Oct 22 13:56:00 BST 2020 armv6l GNU/Linux
>
> root@pih:~# btrfs --version
> btrfs-progs v4.20.1
>
> root@pih:~# btrfs fi show
> Label: 'DATA' uuid: 6be1e09c-d1a5-469d-932b-a8d1c339afae
> Total devices 1 FS bytes used 377.57GiB
> devid 2 size 931.51GiB used 383.06GiB path
> /dev/mapper/luks_open_backup0
>
> root@pih:~# btrfs fi df /mnt/backup
> Data, single: total=375.00GiB, used=374.25GiB
> System, DUP: total=32.00MiB, used=64.00KiB
> Metadata, DUP: total=4.00GiB, used=3.32GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
>
> dmesg is empty for the time of import/btrfs receive.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-21 4:40 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-17 18:49 received uuid not set btrfs send/receive Anders Halman
2021-01-17 19:21 ` Hugo Mills
2021-01-17 21:07 ` Chris Murphy
2021-01-20 3:19 ` Anders Halman
2021-01-21 3:44 ` [SOLVED] " Anders Halman
2021-01-18 5:49 ` Andrei Borzenkov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).