* btrfs receive fails with "failed to clone extents" @ 2021-09-22 19:37 Yuxuan Shui 2021-09-22 23:24 ` Qu Wenruo 2021-09-23 10:13 ` Filipe Manana 0 siblings, 2 replies; 19+ messages in thread From: Yuxuan Shui @ 2021-09-22 19:37 UTC (permalink / raw) To: linux-btrfs Hi, The problem is as the title states. Relevant logs from `btrfs receive -vvv`: mkfile o119493905-1537066-0 rename o119493905-1537066-0 -> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h source offset=0 offset=0 length=131072 ERROR: failed to clone extents to shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: Invalid argument stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, on the receiving end: File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h Size: 145904 Blocks: 288 IO Block: 4096 regular file Looks to me the range of clone is within the boundary of the source file. Not sure why this failed? Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 and btrfs-progs 5.13.1 -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-22 19:37 btrfs receive fails with "failed to clone extents" Yuxuan Shui @ 2021-09-22 23:24 ` Qu Wenruo 2021-09-23 1:34 ` Yuxuan Shui 2021-09-23 10:13 ` Filipe Manana 1 sibling, 1 reply; 19+ messages in thread From: Qu Wenruo @ 2021-09-22 23:24 UTC (permalink / raw) To: Yuxuan Shui, linux-btrfs On 2021/9/23 03:37, Yuxuan Shui wrote: > Hi, > > The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > > mkfile o119493905-1537066-0 > rename o119493905-1537066-0 -> > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > source offset=0 offset=0 length=131072 > ERROR: failed to clone extents to > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > Invalid argument > > stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > on the receiving end: > > File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > Size: 145904 Blocks: 288 IO Block: 4096 regular file > > Looks to me the range of clone is within the boundary of the source > file. Not sure why this failed? The most common reason is, you have changed the parent subvolume from RO to RW, and modified the parent subvolume, then converted it back to RO. Btrfs should not allow such incremental send at all. We're already working on such problem, but next time if you want to modify a RO subvolume which could be the parent subvolume of incremental send, please either do a snapshot then modify the snapshot, or just don't do it. Thanks, Qu > > Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > and btrfs-progs 5.13.1 > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-22 23:24 ` Qu Wenruo @ 2021-09-23 1:34 ` Yuxuan Shui 2021-09-23 1:40 ` Yuxuan Shui 2021-09-23 9:27 ` Graham Cobb 0 siblings, 2 replies; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 1:34 UTC (permalink / raw) To: Qu Wenruo; +Cc: linux-btrfs Hi, On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2021/9/23 03:37, Yuxuan Shui wrote: > > Hi, > > > > The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > > > > mkfile o119493905-1537066-0 > > rename o119493905-1537066-0 -> > > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > > utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > > clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > > - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > > source offset=0 offset=0 length=131072 > > ERROR: failed to clone extents to > > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > > Invalid argument > > > > stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > > on the receiving end: > > > > File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > > Size: 145904 Blocks: 288 IO Block: 4096 regular file > > > > Looks to me the range of clone is within the boundary of the source > > file. Not sure why this failed? > > The most common reason is, you have changed the parent subvolume from RO > to RW, and modified the parent subvolume, then converted it back to RO. This is 100% not the case. I created these snapshots as RO right before sending, and definitely haven't changed them to RW ever. > > Btrfs should not allow such incremental send at all. > > We're already working on such problem, but next time if you want to > modify a RO subvolume which could be the parent subvolume of incremental > send, please either do a snapshot then modify the snapshot, or just > don't do it. > > Thanks, > Qu > > > > > Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > > and btrfs-progs 5.13.1 > > -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 1:34 ` Yuxuan Shui @ 2021-09-23 1:40 ` Yuxuan Shui 2021-09-23 2:32 ` Qu Wenruo 2021-09-23 9:27 ` Graham Cobb 1 sibling, 1 reply; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 1:40 UTC (permalink / raw) To: Qu Wenruo; +Cc: linux-btrfs On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > > Hi, > > On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > > > > > On 2021/9/23 03:37, Yuxuan Shui wrote: > > > Hi, > > > > > > The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > > > > > > mkfile o119493905-1537066-0 > > > rename o119493905-1537066-0 -> > > > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > > > utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > > > clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > > > - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > > > source offset=0 offset=0 length=131072 > > > ERROR: failed to clone extents to > > > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > > > Invalid argument > > > > > > stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > > > on the receiving end: > > > > > > File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > > > Size: 145904 Blocks: 288 IO Block: 4096 regular file > > > > > > Looks to me the range of clone is within the boundary of the source > > > file. Not sure why this failed? > > > > The most common reason is, you have changed the parent subvolume from RO > > to RW, and modified the parent subvolume, then converted it back to RO. > > This is 100% not the case. I created these snapshots as RO right > before sending, and definitely haven't > changed them to RW ever. Besides that, I straced the btrfs command and this clone ioctl definitely looks valid, irregardless of whether the parent snapshot has been changed or not. The length looks to be aligned (128k), and the range is within the source file. Why did the clone fail? > > > > > Btrfs should not allow such incremental send at all. > > > > We're already working on such problem, but next time if you want to > > modify a RO subvolume which could be the parent subvolume of incremental > > send, please either do a snapshot then modify the snapshot, or just > > don't do it. > > > > Thanks, > > Qu > > > > > > > > Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > > > and btrfs-progs 5.13.1 > > > > > > > -- > > Regards > Yuxuan Shui -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 1:40 ` Yuxuan Shui @ 2021-09-23 2:32 ` Qu Wenruo [not found] ` <CAGqt0zz+=nUYbNwLSPYwzYcNLgyxsWT22p+jwwFpAOcyAHAWgA@mail.gmail.com> 0 siblings, 1 reply; 19+ messages in thread From: Qu Wenruo @ 2021-09-23 2:32 UTC (permalink / raw) To: Yuxuan Shui; +Cc: linux-btrfs On 2021/9/23 09:40, Yuxuan Shui wrote: > On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: >> >> Hi, >> >> On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>> >>> >>> >>> On 2021/9/23 03:37, Yuxuan Shui wrote: >>>> Hi, >>>> >>>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: >>>> >>>> mkfile o119493905-1537066-0 >>>> rename o119493905-1537066-0 -> >>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h >>>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include >>>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h >>>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h >>>> source offset=0 offset=0 length=131072 >>>> ERROR: failed to clone extents to >>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: >>>> Invalid argument >>>> >>>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, >>>> on the receiving end: >>>> >>>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h >>>> Size: 145904 Blocks: 288 IO Block: 4096 regular file >>>> >>>> Looks to me the range of clone is within the boundary of the source >>>> file. Not sure why this failed? >>> >>> The most common reason is, you have changed the parent subvolume from RO >>> to RW, and modified the parent subvolume, then converted it back to RO. >> >> This is 100% not the case. I created these snapshots as RO right >> before sending, and definitely haven't >> changed them to RW ever. > > Besides that, I straced the btrfs command and this clone ioctl > definitely looks valid, irregardless of whether the parent snapshot > has been changed or not. The length looks to be aligned (128k), and > the range is within the source file. Why did the clone fail? The clone source must not have certain bits like NODATACOW. If non-incremental send stream works, then it's almost certain it's the received UUID bug we're working on. Thanks, Qu > >> >>> >>> Btrfs should not allow such incremental send at all. >>> >>> We're already working on such problem, but next time if you want to >>> modify a RO subvolume which could be the parent subvolume of incremental >>> send, please either do a snapshot then modify the snapshot, or just >>> don't do it. >>> >>> Thanks, >>> Qu >>> >>>> >>>> Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 >>>> and btrfs-progs 5.13.1 >>>> >> >> >> >> -- >> >> Regards >> Yuxuan Shui > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <CAGqt0zz+=nUYbNwLSPYwzYcNLgyxsWT22p+jwwFpAOcyAHAWgA@mail.gmail.com>]
* Fwd: btrfs receive fails with "failed to clone extents" [not found] ` <CAGqt0zz+=nUYbNwLSPYwzYcNLgyxsWT22p+jwwFpAOcyAHAWgA@mail.gmail.com> @ 2021-09-23 9:27 ` Yuxuan Shui [not found] ` <e83029f0-8583-91b9-47c8-a99d4b00a6ae@gmx.com> 1 sibling, 0 replies; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 9:27 UTC (permalink / raw) To: linux-btrfs Hi, (Sorry I forgot to CC linux-btrfs) On Thu, Sep 23, 2021 at 3:32 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2021/9/23 09:40, Yuxuan Shui wrote: > > On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > >> > >> Hi, > >> > >> On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>> > >>> > >>> > >>> On 2021/9/23 03:37, Yuxuan Shui wrote: > >>>> Hi, > >>>> > >>>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > >>>> > >>>> mkfile o119493905-1537066-0 > >>>> rename o119493905-1537066-0 -> > >>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > >>>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>>> source offset=0 offset=0 length=131072 > >>>> ERROR: failed to clone extents to > >>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > >>>> Invalid argument > >>>> > >>>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > >>>> on the receiving end: > >>>> > >>>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>>> Size: 145904 Blocks: 288 IO Block: 4096 regular file > >>>> > >>>> Looks to me the range of clone is within the boundary of the source > >>>> file. Not sure why this failed? > >>> > >>> The most common reason is, you have changed the parent subvolume from RO > >>> to RW, and modified the parent subvolume, then converted it back to RO. > >> > >> This is 100% not the case. I created these snapshots as RO right > >> before sending, and definitely haven't > >> changed them to RW ever. > > > > Besides that, I straced the btrfs command and this clone ioctl > > definitely looks valid, irregardless of whether the parent snapshot > > has been changed or not. The length looks to be aligned (128k), and > > the range is within the source file. Why did the clone fail? > > The clone source must not have certain bits like NODATACOW. lsattr doesn't show anything. The entire file system is mounted with nodatasum, though. But I assume if this bit is the problem, send won't fail just on this particular file? > > If non-incremental send stream works, then it's almost certain it's the > received UUID bug we're working on. I haven't tried non-incremental. If by received UUID bug you meant this one: https://lore.kernel.org/linux-btrfs/87blnsuv7m.fsf@gmail.com/T/ , then I don't think this is the one. I don't have duplicated received UUIDs on either end. > > Thanks, > Qu > > > > >> > >>> > >>> Btrfs should not allow such incremental send at all. > >>> > >>> We're already working on such problem, but next time if you want to > >>> modify a RO subvolume which could be the parent subvolume of incremental > >>> send, please either do a snapshot then modify the snapshot, or just > >>> don't do it. > >>> > >>> Thanks, > >>> Qu > >>> > >>>> > >>>> Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > >>>> and btrfs-progs 5.13.1 > >>>> > >> > >> > >> > >> -- > >> > >> Regards > >> Yuxuan Shui > > > > > > -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <e83029f0-8583-91b9-47c8-a99d4b00a6ae@gmx.com>]
* Re: btrfs receive fails with "failed to clone extents" [not found] ` <e83029f0-8583-91b9-47c8-a99d4b00a6ae@gmx.com> @ 2021-09-23 9:38 ` Yuxuan Shui 2021-09-23 9:43 ` Yuxuan Shui 0 siblings, 1 reply; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 9:38 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs Hi, On Thu, Sep 23, 2021 at 4:28 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2021/9/23 11:09, Yuxuan Shui wrote: > > Hi, > > > > On Thu, Sep 23, 2021 at 3:32 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> > >> > >> > >> On 2021/9/23 09:40, Yuxuan Shui wrote: > >>> On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > >>>> > >>>> Hi, > >>>> > >>>> On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>>>> > >>>>> > >>>>> > >>>>> On 2021/9/23 03:37, Yuxuan Shui wrote: > >>>>>> Hi, > >>>>>> > >>>>>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > >>>>>> > >>>>>> mkfile o119493905-1537066-0 > >>>>>> rename o119493905-1537066-0 -> > >>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>>>>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > >>>>>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>>>>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>>>>> source offset=0 offset=0 length=131072 > >>>>>> ERROR: failed to clone extents to > >>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > >>>>>> Invalid argument > >>>>>> > >>>>>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > >>>>>> on the receiving end: > >>>>>> > >>>>>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>>>>> Size: 145904 Blocks: 288 IO Block: 4096 regular file > >>>>>> > >>>>>> Looks to me the range of clone is within the boundary of the source > >>>>>> file. Not sure why this failed? > >>>>> > >>>>> The most common reason is, you have changed the parent subvolume from RO > >>>>> to RW, and modified the parent subvolume, then converted it back to RO. > >>>> > >>>> This is 100% not the case. I created these snapshots as RO right > >>>> before sending, and definitely haven't > >>>> changed them to RW ever. > >>> > >>> Besides that, I straced the btrfs command and this clone ioctl > >>> definitely looks valid, irregardless of whether the parent snapshot > >>> has been changed or not. The length looks to be aligned (128k), and > >>> the range is within the source file. Why did the clone fail? > >> > >> The clone source must not have certain bits like NODATACOW. > > > > lsattr doesn't show anything. The entire file system is mounted with > > nodatasum, though. But I assume if this bit is the problem, send won't > > fail just on this particular file? > > If source and dest file has different NODATASUM flags (one has and the > other doesn't), then it will also be rejected. > > NODATASUM flag won't show up in lsattr, thus you need to use "btrfs > prop" to check. I checked but I don't think "btrfs prop" shows the NODATASUM flag. The flags it displays include ro, label and compression only. > > Considering you're mounting with NODATASUM, I guess that could the case. > Your receive target is inheriting the NODATASUM flag, while the source > file doesn't have the NODATASUM flag. Interesting, but the clone is within the same subvolume that is been received. Won't the whole subvolume get the same flag, since the file system is mounted with nodatasum? > > If that's the case, I guess we may hit a new bug for receive, that we > should also update the dest file's flags before doing reflink. > > Could you remove the partially received (and fialed) subvolume, remount > the fs without nodatasum, then try again to see if that's the case? > (It may need to change the destination directory too to remove the > NODATASUM flag) > > Thanks, > Qu > > > >> > >> If non-incremental send stream works, then it's almost certain it's the > >> received UUID bug we're working on. > > > > I haven't tried non-incremental. If by received UUID bug you meant > > this one: https://lore.kernel.org/linux-btrfs/87blnsuv7m.fsf@gmail.com/T/ > > , then I don't think this is the one. I don't have duplicated received > > UUIDs on either end. > > > >> > >> Thanks, > >> Qu > >> > >>> > >>>> > >>>>> > >>>>> Btrfs should not allow such incremental send at all. > >>>>> > >>>>> We're already working on such problem, but next time if you want to > >>>>> modify a RO subvolume which could be the parent subvolume of incremental > >>>>> send, please either do a snapshot then modify the snapshot, or just > >>>>> don't do it. > >>>>> > >>>>> Thanks, > >>>>> Qu > >>>>> > >>>>>> > >>>>>> Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > >>>>>> and btrfs-progs 5.13.1 > >>>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> > >>>> Regards > >>>> Yuxuan Shui > >>> > >>> > >>> > > > > > > -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 9:38 ` Yuxuan Shui @ 2021-09-23 9:43 ` Yuxuan Shui 2021-09-23 9:44 ` Qu Wenruo 0 siblings, 1 reply; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 9:43 UTC (permalink / raw) To: Qu Wenruo, linux-btrfs Hi, On Thu, Sep 23, 2021 at 10:38 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > > Hi, > > On Thu, Sep 23, 2021 at 4:28 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > > > > > On 2021/9/23 11:09, Yuxuan Shui wrote: > > > Hi, > > > > > > On Thu, Sep 23, 2021 at 3:32 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > >> > > >> > > >> > > >> On 2021/9/23 09:40, Yuxuan Shui wrote: > > >>> On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > > >>>> > > >>>> Hi, > > >>>> > > >>>> On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > >>>>> > > >>>>> > > >>>>> > > >>>>> On 2021/9/23 03:37, Yuxuan Shui wrote: > > >>>>>> Hi, > > >>>>>> > > >>>>>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > > >>>>>> > > >>>>>> mkfile o119493905-1537066-0 > > >>>>>> rename o119493905-1537066-0 -> > > >>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > > >>>>>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > > >>>>>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > > >>>>>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > > >>>>>> source offset=0 offset=0 length=131072 > > >>>>>> ERROR: failed to clone extents to > > >>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > > >>>>>> Invalid argument > > >>>>>> > > >>>>>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > > >>>>>> on the receiving end: > > >>>>>> > > >>>>>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > > >>>>>> Size: 145904 Blocks: 288 IO Block: 4096 regular file > > >>>>>> > > >>>>>> Looks to me the range of clone is within the boundary of the source > > >>>>>> file. Not sure why this failed? > > >>>>> > > >>>>> The most common reason is, you have changed the parent subvolume from RO > > >>>>> to RW, and modified the parent subvolume, then converted it back to RO. > > >>>> > > >>>> This is 100% not the case. I created these snapshots as RO right > > >>>> before sending, and definitely haven't > > >>>> changed them to RW ever. > > >>> > > >>> Besides that, I straced the btrfs command and this clone ioctl > > >>> definitely looks valid, irregardless of whether the parent snapshot > > >>> has been changed or not. The length looks to be aligned (128k), and > > >>> the range is within the source file. Why did the clone fail? > > >> > > >> The clone source must not have certain bits like NODATACOW. > > > > > > lsattr doesn't show anything. The entire file system is mounted with > > > nodatasum, though. But I assume if this bit is the problem, send won't > > > fail just on this particular file? > > > > If source and dest file has different NODATASUM flags (one has and the > > other doesn't), then it will also be rejected. > > > > NODATASUM flag won't show up in lsattr, thus you need to use "btrfs > > prop" to check. > > I checked but I don't think "btrfs prop" shows the NODATASUM flag. The > flags it displays include ro, label and compression only. > > > > > Considering you're mounting with NODATASUM, I guess that could the case. > > Your receive target is inheriting the NODATASUM flag, while the source > > file doesn't have the NODATASUM flag. > > > Interesting, but the clone is within the same subvolume that is been > received. Won't the whole subvolume get the same flag, since the file > system is mounted with nodatasum? > > > > > If that's the case, I guess we may hit a new bug for receive, that we > > should also update the dest file's flags before doing reflink. > > > > Could you remove the partially received (and fialed) subvolume, remount > > the fs without nodatasum, then try again to see if that's the case? > > (It may need to change the destination directory too to remove the > > NODATASUM flag) Yes, the send/receive worked without nodatasum. > > > > Thanks, > > Qu > > > > > >> > > >> If non-incremental send stream works, then it's almost certain it's the > > >> received UUID bug we're working on. > > > > > > I haven't tried non-incremental. If by received UUID bug you meant > > > this one: https://lore.kernel.org/linux-btrfs/87blnsuv7m.fsf@gmail.com/T/ > > > , then I don't think this is the one. I don't have duplicated received > > > UUIDs on either end. > > > > > >> > > >> Thanks, > > >> Qu > > >> > > >>> > > >>>> > > >>>>> > > >>>>> Btrfs should not allow such incremental send at all. > > >>>>> > > >>>>> We're already working on such problem, but next time if you want to > > >>>>> modify a RO subvolume which could be the parent subvolume of incremental > > >>>>> send, please either do a snapshot then modify the snapshot, or just > > >>>>> don't do it. > > >>>>> > > >>>>> Thanks, > > >>>>> Qu > > >>>>> > > >>>>>> > > >>>>>> Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > > >>>>>> and btrfs-progs 5.13.1 > > >>>>>> > > >>>> > > >>>> > > >>>> > > >>>> -- > > >>>> > > >>>> Regards > > >>>> Yuxuan Shui > > >>> > > >>> > > >>> > > > > > > > > > > > > > -- > > Regards > Yuxuan Shui -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 9:43 ` Yuxuan Shui @ 2021-09-23 9:44 ` Qu Wenruo 2021-09-23 10:08 ` Yuxuan Shui 0 siblings, 1 reply; 19+ messages in thread From: Qu Wenruo @ 2021-09-23 9:44 UTC (permalink / raw) To: Yuxuan Shui, Qu Wenruo, linux-btrfs On 2021/9/23 17:43, Yuxuan Shui wrote: > Hi, > > On Thu, Sep 23, 2021 at 10:38 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: >> >> Hi, >> >> On Thu, Sep 23, 2021 at 4:28 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>> >>> >>> >>> On 2021/9/23 11:09, Yuxuan Shui wrote: >>>> Hi, >>>> >>>> On Thu, Sep 23, 2021 at 3:32 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>>> >>>>> >>>>> >>>>> On 2021/9/23 09:40, Yuxuan Shui wrote: >>>>>> On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 2021/9/23 03:37, Yuxuan Shui wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: >>>>>>>>> >>>>>>>>> mkfile o119493905-1537066-0 >>>>>>>>> rename o119493905-1537066-0 -> >>>>>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h >>>>>>>>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include >>>>>>>>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h >>>>>>>>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h >>>>>>>>> source offset=0 offset=0 length=131072 >>>>>>>>> ERROR: failed to clone extents to >>>>>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: >>>>>>>>> Invalid argument >>>>>>>>> >>>>>>>>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, >>>>>>>>> on the receiving end: >>>>>>>>> >>>>>>>>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h >>>>>>>>> Size: 145904 Blocks: 288 IO Block: 4096 regular file >>>>>>>>> >>>>>>>>> Looks to me the range of clone is within the boundary of the source >>>>>>>>> file. Not sure why this failed? >>>>>>>> >>>>>>>> The most common reason is, you have changed the parent subvolume from RO >>>>>>>> to RW, and modified the parent subvolume, then converted it back to RO. >>>>>>> >>>>>>> This is 100% not the case. I created these snapshots as RO right >>>>>>> before sending, and definitely haven't >>>>>>> changed them to RW ever. >>>>>> >>>>>> Besides that, I straced the btrfs command and this clone ioctl >>>>>> definitely looks valid, irregardless of whether the parent snapshot >>>>>> has been changed or not. The length looks to be aligned (128k), and >>>>>> the range is within the source file. Why did the clone fail? >>>>> >>>>> The clone source must not have certain bits like NODATACOW. >>>> >>>> lsattr doesn't show anything. The entire file system is mounted with >>>> nodatasum, though. But I assume if this bit is the problem, send won't >>>> fail just on this particular file? >>> >>> If source and dest file has different NODATASUM flags (one has and the >>> other doesn't), then it will also be rejected. >>> >>> NODATASUM flag won't show up in lsattr, thus you need to use "btrfs >>> prop" to check. >> >> I checked but I don't think "btrfs prop" shows the NODATASUM flag. The >> flags it displays include ro, label and compression only. >> >>> >>> Considering you're mounting with NODATASUM, I guess that could the case. >>> Your receive target is inheriting the NODATASUM flag, while the source >>> file doesn't have the NODATASUM flag. >> >> >> Interesting, but the clone is within the same subvolume that is been >> received. Won't the whole subvolume get the same flag, since the file >> system is mounted with nodatasum? >> >>> >>> If that's the case, I guess we may hit a new bug for receive, that we >>> should also update the dest file's flags before doing reflink. >>> >>> Could you remove the partially received (and fialed) subvolume, remount >>> the fs without nodatasum, then try again to see if that's the case? >>> (It may need to change the destination directory too to remove the >>> NODATASUM flag) > > Yes, the send/receive worked without nodatasum. Mind to provide the full send stream dump? This indeed looks like a bug now. Thanks, Qu > >>> >>> Thanks, >>> Qu >>>> >>>>> >>>>> If non-incremental send stream works, then it's almost certain it's the >>>>> received UUID bug we're working on. >>>> >>>> I haven't tried non-incremental. If by received UUID bug you meant >>>> this one: https://lore.kernel.org/linux-btrfs/87blnsuv7m.fsf@gmail.com/T/ >>>> , then I don't think this is the one. I don't have duplicated received >>>> UUIDs on either end. >>>> >>>>> >>>>> Thanks, >>>>> Qu >>>>> >>>>>> >>>>>>> >>>>>>>> >>>>>>>> Btrfs should not allow such incremental send at all. >>>>>>>> >>>>>>>> We're already working on such problem, but next time if you want to >>>>>>>> modify a RO subvolume which could be the parent subvolume of incremental >>>>>>>> send, please either do a snapshot then modify the snapshot, or just >>>>>>>> don't do it. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Qu >>>>>>>> >>>>>>>>> >>>>>>>>> Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 >>>>>>>>> and btrfs-progs 5.13.1 >>>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> Regards >>>>>>> Yuxuan Shui >>>>>> >>>>>> >>>>>> >>>> >>>> >>>> >> >> >> >> -- >> >> Regards >> Yuxuan Shui > > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 9:44 ` Qu Wenruo @ 2021-09-23 10:08 ` Yuxuan Shui 2021-09-23 10:19 ` Qu Wenruo 0 siblings, 1 reply; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 10:08 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs Hi, On Thu, Sep 23, 2021 at 10:45 AM Qu Wenruo <wqu@suse.com> wrote: > > > > On 2021/9/23 17:43, Yuxuan Shui wrote: > > Hi, > > > > On Thu, Sep 23, 2021 at 10:38 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > >> > >> Hi, > >> > >> On Thu, Sep 23, 2021 at 4:28 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>> > >>> > >>> > >>> On 2021/9/23 11:09, Yuxuan Shui wrote: > >>>> Hi, > >>>> > >>>> On Thu, Sep 23, 2021 at 3:32 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>>>> > >>>>> > >>>>> > >>>>> On 2021/9/23 09:40, Yuxuan Shui wrote: > >>>>>> On Thu, Sep 23, 2021 at 2:34 AM Yuxuan Shui <yshuiv7@gmail.com> wrote: > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 2021/9/23 03:37, Yuxuan Shui wrote: > >>>>>>>>> Hi, > >>>>>>>>> > >>>>>>>>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > >>>>>>>>> > >>>>>>>>> mkfile o119493905-1537066-0 > >>>>>>>>> rename o119493905-1537066-0 -> > >>>>>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>>>>>>>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > >>>>>>>>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>>>>>>>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>>>>>>>> source offset=0 offset=0 length=131072 > >>>>>>>>> ERROR: failed to clone extents to > >>>>>>>>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > >>>>>>>>> Invalid argument > >>>>>>>>> > >>>>>>>>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > >>>>>>>>> on the receiving end: > >>>>>>>>> > >>>>>>>>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>>>>>>>> Size: 145904 Blocks: 288 IO Block: 4096 regular file > >>>>>>>>> > >>>>>>>>> Looks to me the range of clone is within the boundary of the source > >>>>>>>>> file. Not sure why this failed? > >>>>>>>> > >>>>>>>> The most common reason is, you have changed the parent subvolume from RO > >>>>>>>> to RW, and modified the parent subvolume, then converted it back to RO. > >>>>>>> > >>>>>>> This is 100% not the case. I created these snapshots as RO right > >>>>>>> before sending, and definitely haven't > >>>>>>> changed them to RW ever. > >>>>>> > >>>>>> Besides that, I straced the btrfs command and this clone ioctl > >>>>>> definitely looks valid, irregardless of whether the parent snapshot > >>>>>> has been changed or not. The length looks to be aligned (128k), and > >>>>>> the range is within the source file. Why did the clone fail? > >>>>> > >>>>> The clone source must not have certain bits like NODATACOW. > >>>> > >>>> lsattr doesn't show anything. The entire file system is mounted with > >>>> nodatasum, though. But I assume if this bit is the problem, send won't > >>>> fail just on this particular file? > >>> > >>> If source and dest file has different NODATASUM flags (one has and the > >>> other doesn't), then it will also be rejected. > >>> > >>> NODATASUM flag won't show up in lsattr, thus you need to use "btrfs > >>> prop" to check. > >> > >> I checked but I don't think "btrfs prop" shows the NODATASUM flag. The > >> flags it displays include ro, label and compression only. > >> > >>> > >>> Considering you're mounting with NODATASUM, I guess that could the case. > >>> Your receive target is inheriting the NODATASUM flag, while the source > >>> file doesn't have the NODATASUM flag. > >> > >> > >> Interesting, but the clone is within the same subvolume that is been > >> received. Won't the whole subvolume get the same flag, since the file > >> system is mounted with nodatasum? > >> > >>> > >>> If that's the case, I guess we may hit a new bug for receive, that we > >>> should also update the dest file's flags before doing reflink. > >>> > >>> Could you remove the partially received (and fialed) subvolume, remount > >>> the fs without nodatasum, then try again to see if that's the case? > >>> (It may need to change the destination directory too to remove the > >>> NODATASUM flag) > > > > Yes, the send/receive worked without nodatasum. > > Mind to provide the full send stream dump? > > This indeed looks like a bug now. Sorry, this might have data I am not allowed to share, and I don't think I will be able to vet all the files in the stream. But if you can let me know what I can do to help debugging, I could try that. > > Thanks, > Qu > > > > >>> > >>> Thanks, > >>> Qu > >>>> > >>>>> > >>>>> If non-incremental send stream works, then it's almost certain it's the > >>>>> received UUID bug we're working on. > >>>> > >>>> I haven't tried non-incremental. If by received UUID bug you meant > >>>> this one: https://lore.kernel.org/linux-btrfs/87blnsuv7m.fsf@gmail.com/T/ > >>>> , then I don't think this is the one. I don't have duplicated received > >>>> UUIDs on either end. > >>>> > >>>>> > >>>>> Thanks, > >>>>> Qu > >>>>> > >>>>>> > >>>>>>> > >>>>>>>> > >>>>>>>> Btrfs should not allow such incremental send at all. > >>>>>>>> > >>>>>>>> We're already working on such problem, but next time if you want to > >>>>>>>> modify a RO subvolume which could be the parent subvolume of incremental > >>>>>>>> send, please either do a snapshot then modify the snapshot, or just > >>>>>>>> don't do it. > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> Qu > >>>>>>>> > >>>>>>>>> > >>>>>>>>> Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > >>>>>>>>> and btrfs-progs 5.13.1 > >>>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> > >>>>>>> Regards > >>>>>>> Yuxuan Shui > >>>>>> > >>>>>> > >>>>>> > >>>> > >>>> > >>>> > >> > >> > >> > >> -- > >> > >> Regards > >> Yuxuan Shui > > > > > > > -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 10:08 ` Yuxuan Shui @ 2021-09-23 10:19 ` Qu Wenruo 2021-09-23 10:46 ` Filipe Manana 0 siblings, 1 reply; 19+ messages in thread From: Qu Wenruo @ 2021-09-23 10:19 UTC (permalink / raw) To: Yuxuan Shui; +Cc: Qu Wenruo, linux-btrfs On 2021/9/23 18:08, Yuxuan Shui wrote: > Hi, > > On Thu, Sep 23, 2021 at 10:45 AM Qu Wenruo <wqu@suse.com> wrote: >> >> Mind to provide the full send stream dump? >> >> This indeed looks like a bug now. > > Sorry, this might have data I am not allowed to share, and I don't > think I will be able to vet all the files in the stream. > > But if you can let me know what I can do to help debugging, I could try that. That's totally understandable. Surprisingly, I can't even find a proper way to get the nodatasum flag per-file. Thus I guess things may go complex by using "btrfs ins dump-tree" For the receive failure case, mind to provide the following info? - The inode number of the clone source You can use 'stat' command to get the inode number: $ stat /mnt/btrfs/file | grep Inode Device: 29h/41d Inode: 257 Links: 1 - The tree dump of the clone source $ btrfs ins dump-tree -t <subvolid> <device> | \ grep "(<INODE> INODE_ITEM 0) item" -A 5 item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160 generation 7 transid 7 size 0 nbytes 0 block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 sequence 3 flags 0x0(none) atime 1632391968.333333415 (2021-09-23 18:12:48) ctime 1632391968.333333415 (2021-09-23 18:12:48) <subvolid> is the subvolume id of the clone source. - The tree dump of the clone dest directory The inode number can be get using the same 'stat' command. Then pass it to the same "btrfs ins dump-tree" command, with <subvolid> <device> and <INODE> modified. Then we can be sure what's causing the NODATASUM flag mismatch. Thanks, Qu ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 10:19 ` Qu Wenruo @ 2021-09-23 10:46 ` Filipe Manana 2021-09-23 11:16 ` Qu Wenruo 0 siblings, 1 reply; 19+ messages in thread From: Filipe Manana @ 2021-09-23 10:46 UTC (permalink / raw) To: Qu Wenruo; +Cc: Yuxuan Shui, Qu Wenruo, linux-btrfs On Thu, Sep 23, 2021 at 11:20 AM Qu Wenruo <wqu@suse.com> wrote: > > > > On 2021/9/23 18:08, Yuxuan Shui wrote: > > Hi, > > > > On Thu, Sep 23, 2021 at 10:45 AM Qu Wenruo <wqu@suse.com> wrote: > >> > >> Mind to provide the full send stream dump? > >> > >> This indeed looks like a bug now. > > > > Sorry, this might have data I am not allowed to share, and I don't > > think I will be able to vet all the files in the stream. > > > > But if you can let me know what I can do to help debugging, I could try that. > > That's totally understandable. > > Surprisingly, I can't even find a proper way to get the nodatasum flag > per-file. > Thus I guess things may go complex by using "btrfs ins dump-tree" > > For the receive failure case, mind to provide the following info? > > - The inode number of the clone source > You can use 'stat' command to get the inode number: > $ stat /mnt/btrfs/file | grep Inode > Device: 29h/41d Inode: 257 Links: 1 > > - The tree dump of the clone source > $ btrfs ins dump-tree -t <subvolid> <device> | \ > grep "(<INODE> INODE_ITEM 0) item" -A 5 > item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160 > generation 7 transid 7 size 0 nbytes 0 > block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > sequence 3 flags 0x0(none) > atime 1632391968.333333415 (2021-09-23 18:12:48) > ctime 1632391968.333333415 (2021-09-23 18:12:48) > > <subvolid> is the subvolume id of the clone source. > > - The tree dump of the clone dest directory > The inode number can be get using the same 'stat' command. > Then pass it to the same "btrfs ins dump-tree" command, with > <subvolid> <device> and <INODE> modified. > > Then we can be sure what's causing the NODATASUM flag mismatch. Catching up on the whole thread, that is indeed a possible cause of failure. Someone can do something like: 1) mount fs without -o nodatacow and without -o nodatasum 2) receive snapshot A 3) umount fs 4) mount fs with -o nodatacow or -o nodatasum 5) start receiving an incremental stream that has A as parent and snapshot B as the send snapshot 6) files created in B end up with the NODATASUM flag set 7) the send stream has a clone operation to clone from some file in A to a file in B - this fails, as the file in A does not have the nodatasum bit but the file in B has it I'm thinking that for cases like this, we could use a flag for send to tell it to not generate clone operations and do regular write commands instead. > > Thanks, > Qu > -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 10:46 ` Filipe Manana @ 2021-09-23 11:16 ` Qu Wenruo 2021-09-23 11:24 ` Filipe Manana 2021-09-24 9:47 ` Yuxuan Shui 0 siblings, 2 replies; 19+ messages in thread From: Qu Wenruo @ 2021-09-23 11:16 UTC (permalink / raw) To: fdmanana, Qu Wenruo; +Cc: Yuxuan Shui, linux-btrfs On 2021/9/23 18:46, Filipe Manana wrote: > On Thu, Sep 23, 2021 at 11:20 AM Qu Wenruo <wqu@suse.com> wrote: >> >> >> >> On 2021/9/23 18:08, Yuxuan Shui wrote: >>> Hi, >>> >>> On Thu, Sep 23, 2021 at 10:45 AM Qu Wenruo <wqu@suse.com> wrote: >>>> >>>> Mind to provide the full send stream dump? >>>> >>>> This indeed looks like a bug now. >>> >>> Sorry, this might have data I am not allowed to share, and I don't >>> think I will be able to vet all the files in the stream. >>> >>> But if you can let me know what I can do to help debugging, I could try that. >> >> That's totally understandable. >> >> Surprisingly, I can't even find a proper way to get the nodatasum flag >> per-file. >> Thus I guess things may go complex by using "btrfs ins dump-tree" >> >> For the receive failure case, mind to provide the following info? >> >> - The inode number of the clone source >> You can use 'stat' command to get the inode number: >> $ stat /mnt/btrfs/file | grep Inode >> Device: 29h/41d Inode: 257 Links: 1 >> >> - The tree dump of the clone source >> $ btrfs ins dump-tree -t <subvolid> <device> | \ >> grep "(<INODE> INODE_ITEM 0) item" -A 5 >> item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160 >> generation 7 transid 7 size 0 nbytes 0 >> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 >> sequence 3 flags 0x0(none) >> atime 1632391968.333333415 (2021-09-23 18:12:48) >> ctime 1632391968.333333415 (2021-09-23 18:12:48) >> >> <subvolid> is the subvolume id of the clone source. >> >> - The tree dump of the clone dest directory >> The inode number can be get using the same 'stat' command. >> Then pass it to the same "btrfs ins dump-tree" command, with >> <subvolid> <device> and <INODE> modified. >> >> Then we can be sure what's causing the NODATASUM flag mismatch. > > Catching up on the whole thread, that is indeed a possible cause of failure. > > Someone can do something like: > > 1) mount fs without -o nodatacow and without -o nodatasum > 2) receive snapshot A > 3) umount fs > 4) mount fs with -o nodatacow or -o nodatasum > 5) start receiving an incremental stream that has A as parent and > snapshot B as the send snapshot > 6) files created in B end up with the NODATASUM flag set > 7) the send stream has a clone operation to clone from some file in A > to a file in B - this fails, as the file in A does not have the > nodatasum bit but the file in B has it Exactly. We can craft a test case for it (but without a fix for a while). > > I'm thinking that for cases like this, we could use a flag for send to > tell it to not generate clone operations and do regular write commands > instead. My biggest concern here is, we should properly include the inode flags in the send stream, and provide an interface to change btrfs specific flags (currently only NODATASUM) at the receive side. With that we can set the proper inode flags before writing/cloning, no matter the mount option. But this is a big project, involving changing send stream format, introducing new ioctl interface. Thanks, Qu > >> >> Thanks, >> Qu >> > > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 11:16 ` Qu Wenruo @ 2021-09-23 11:24 ` Filipe Manana 2021-09-24 9:47 ` Yuxuan Shui 1 sibling, 0 replies; 19+ messages in thread From: Filipe Manana @ 2021-09-23 11:24 UTC (permalink / raw) To: Qu Wenruo; +Cc: Qu Wenruo, Yuxuan Shui, linux-btrfs On Thu, Sep 23, 2021 at 12:16 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2021/9/23 18:46, Filipe Manana wrote: > > On Thu, Sep 23, 2021 at 11:20 AM Qu Wenruo <wqu@suse.com> wrote: > >> > >> > >> > >> On 2021/9/23 18:08, Yuxuan Shui wrote: > >>> Hi, > >>> > >>> On Thu, Sep 23, 2021 at 10:45 AM Qu Wenruo <wqu@suse.com> wrote: > >>>> > >>>> Mind to provide the full send stream dump? > >>>> > >>>> This indeed looks like a bug now. > >>> > >>> Sorry, this might have data I am not allowed to share, and I don't > >>> think I will be able to vet all the files in the stream. > >>> > >>> But if you can let me know what I can do to help debugging, I could try that. > >> > >> That's totally understandable. > >> > >> Surprisingly, I can't even find a proper way to get the nodatasum flag > >> per-file. > >> Thus I guess things may go complex by using "btrfs ins dump-tree" > >> > >> For the receive failure case, mind to provide the following info? > >> > >> - The inode number of the clone source > >> You can use 'stat' command to get the inode number: > >> $ stat /mnt/btrfs/file | grep Inode > >> Device: 29h/41d Inode: 257 Links: 1 > >> > >> - The tree dump of the clone source > >> $ btrfs ins dump-tree -t <subvolid> <device> | \ > >> grep "(<INODE> INODE_ITEM 0) item" -A 5 > >> item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160 > >> generation 7 transid 7 size 0 nbytes 0 > >> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > >> sequence 3 flags 0x0(none) > >> atime 1632391968.333333415 (2021-09-23 18:12:48) > >> ctime 1632391968.333333415 (2021-09-23 18:12:48) > >> > >> <subvolid> is the subvolume id of the clone source. > >> > >> - The tree dump of the clone dest directory > >> The inode number can be get using the same 'stat' command. > >> Then pass it to the same "btrfs ins dump-tree" command, with > >> <subvolid> <device> and <INODE> modified. > >> > >> Then we can be sure what's causing the NODATASUM flag mismatch. > > > > Catching up on the whole thread, that is indeed a possible cause of failure. > > > > Someone can do something like: > > > > 1) mount fs without -o nodatacow and without -o nodatasum > > 2) receive snapshot A > > 3) umount fs > > 4) mount fs with -o nodatacow or -o nodatasum > > 5) start receiving an incremental stream that has A as parent and > > snapshot B as the send snapshot > > 6) files created in B end up with the NODATASUM flag set > > 7) the send stream has a clone operation to clone from some file in A > > to a file in B - this fails, as the file in A does not have the > > nodatasum bit but the file in B has it > > Exactly. > > We can craft a test case for it (but without a fix for a while). > > > > > I'm thinking that for cases like this, we could use a flag for send to > > tell it to not generate clone operations and do regular write commands > > instead. > > My biggest concern here is, we should properly include the inode flags > in the send stream, and provide an interface to change btrfs specific > flags (currently only NODATASUM) at the receive side. That would require changing the send stream format. Not sure if Omar's v2 stream allow for that. > > With that we can set the proper inode flags before writing/cloning, no > matter the mount option. Well, making the sending side force the receiving side to have the flags that the sending has (or doesn't have), is not something I'm sure that everyone wants. Compression for example - the sending side has no compression, but the receiving side was mounted with -o compress because it wants compression. Having the send side clear the compression flag on every file would be surprising to many users. The same may or may not apply to nodatacow or nodatasum. I think there are people relying on the mount options of the receiving side taking effect. > > But this is a big project, involving changing send stream format, > introducing new ioctl interface. > > Thanks, > Qu > > > > >> > >> Thanks, > >> Qu > >> > > > > -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 11:16 ` Qu Wenruo 2021-09-23 11:24 ` Filipe Manana @ 2021-09-24 9:47 ` Yuxuan Shui 2021-09-24 9:57 ` Qu Wenruo 1 sibling, 1 reply; 19+ messages in thread From: Yuxuan Shui @ 2021-09-24 9:47 UTC (permalink / raw) To: Qu Wenruo; +Cc: fdmanana, Qu Wenruo, linux-btrfs Hello, On Thu, Sep 23, 2021 at 12:16 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > > > > On 2021/9/23 18:46, Filipe Manana wrote: > > On Thu, Sep 23, 2021 at 11:20 AM Qu Wenruo <wqu@suse.com> wrote: > >> > >> > >> > >> On 2021/9/23 18:08, Yuxuan Shui wrote: > >>> Hi, > >>> > >>> On Thu, Sep 23, 2021 at 10:45 AM Qu Wenruo <wqu@suse.com> wrote: > >>>> > >>>> Mind to provide the full send stream dump? > >>>> > >>>> This indeed looks like a bug now. > >>> > >>> Sorry, this might have data I am not allowed to share, and I don't > >>> think I will be able to vet all the files in the stream. > >>> > >>> But if you can let me know what I can do to help debugging, I could try that. > >> > >> That's totally understandable. > >> > >> Surprisingly, I can't even find a proper way to get the nodatasum flag > >> per-file. > >> Thus I guess things may go complex by using "btrfs ins dump-tree" > >> > >> For the receive failure case, mind to provide the following info? > >> > >> - The inode number of the clone source > >> You can use 'stat' command to get the inode number: > >> $ stat /mnt/btrfs/file | grep Inode > >> Device: 29h/41d Inode: 257 Links: 1 > >> > >> - The tree dump of the clone source > >> $ btrfs ins dump-tree -t <subvolid> <device> | \ > >> grep "(<INODE> INODE_ITEM 0) item" -A 5 > >> item 4 key (257 INODE_ITEM 0) itemoff 15883 itemsize 160 > >> generation 7 transid 7 size 0 nbytes 0 > >> block group 0 mode 100644 links 1 uid 0 gid 0 rdev 0 > >> sequence 3 flags 0x0(none) > >> atime 1632391968.333333415 (2021-09-23 18:12:48) > >> ctime 1632391968.333333415 (2021-09-23 18:12:48) > >> > >> <subvolid> is the subvolume id of the clone source. > >> > >> - The tree dump of the clone dest directory > >> The inode number can be get using the same 'stat' command. > >> Then pass it to the same "btrfs ins dump-tree" command, with > >> <subvolid> <device> and <INODE> modified. > >> > >> Then we can be sure what's causing the NODATASUM flag mismatch. > > > > Catching up on the whole thread, that is indeed a possible cause of failure. > > > > Someone can do something like: > > > > 1) mount fs without -o nodatacow and without -o nodatasum > > 2) receive snapshot A > > 3) umount fs > > 4) mount fs with -o nodatacow or -o nodatasum > > 5) start receiving an incremental stream that has A as parent and > > snapshot B as the send snapshot > > 6) files created in B end up with the NODATASUM flag set > > 7) the send stream has a clone operation to clone from some file in A > > to a file in B - this fails, as the file in A does not have the > > nodatasum bit but the file in B has it > > Exactly. > > We can craft a test case for it (but without a fix for a while). As a quick fix, is it possible to convert existing files that are not nodatasum to nodatasum? > > > > > I'm thinking that for cases like this, we could use a flag for send to > > tell it to not generate clone operations and do regular write commands > > instead. > > My biggest concern here is, we should properly include the inode flags > in the send stream, and provide an interface to change btrfs specific > flags (currently only NODATASUM) at the receive side. > > With that we can set the proper inode flags before writing/cloning, no > matter the mount option. > > But this is a big project, involving changing send stream format, > introducing new ioctl interface. > > Thanks, > Qu > > > > >> > >> Thanks, > >> Qu > >> > > > > I have a dumb question. As an outsider, it seems to me that cloning from a file with datasum to a file with nodatasum can have a well defined semantic? i.e. say I clone from A (datasum) to B (nodatasum), then reading from B should skip checksum verification, and modifying B would write data without checksum, and leaving A with checksum. Is this not practical to implement due to btrfs' internal structure? Thanks! -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-24 9:47 ` Yuxuan Shui @ 2021-09-24 9:57 ` Qu Wenruo 0 siblings, 0 replies; 19+ messages in thread From: Qu Wenruo @ 2021-09-24 9:57 UTC (permalink / raw) To: Yuxuan Shui; +Cc: fdmanana, Qu Wenruo, linux-btrfs On 2021/9/24 17:47, Yuxuan Shui wrote: > Hello, > > On Thu, Sep 23, 2021 at 12:16 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> [...] >> >> Exactly. >> >> We can craft a test case for it (but without a fix for a while). > > As a quick fix, is it possible to convert existing files that are not > nodatasum to nodatasum? Well, this is embarrassing now, I don't even know a way to show/set per-file NODATASUM flag. So I'm afraid it's not possible yet. > >> >>> >>> I'm thinking that for cases like this, we could use a flag for send to >>> tell it to not generate clone operations and do regular write commands >>> instead. >> >> My biggest concern here is, we should properly include the inode flags >> in the send stream, and provide an interface to change btrfs specific >> flags (currently only NODATASUM) at the receive side. >> >> With that we can set the proper inode flags before writing/cloning, no >> matter the mount option. >> >> But this is a big project, involving changing send stream format, >> introducing new ioctl interface. >> >> Thanks, >> Qu >> >>> >>>> >>>> Thanks, >>>> Qu >>>> >>> >>> > > I have a dumb question. As an outsider, it seems to me that cloning > from a file with datasum to a file with nodatasum can have a well > defined semantic? i.e. say I clone from A (datasum) to B (nodatasum), > then reading from B should skip checksum verification, and modifying B > would write data without checksum, and leaving A with checksum. Clone means it will share the on-disk data, without doing real write. Then the problem is, one inode says the data should has data checksum, the other says the same data should has no data checksum. This inconsistent status is prevent us from cloning in the first place. > > Is this not practical to implement due to btrfs' internal structure? I discussed with Filipe, we're more or less fine to fix it in btrfs-progs (receive side) by fallback back to buffered write (read the data from the inode, write it back to the destination). By this, we can workaround the problem as a quick (and maybe final) fix. Thanks, Qu > > Thanks! > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 1:34 ` Yuxuan Shui 2021-09-23 1:40 ` Yuxuan Shui @ 2021-09-23 9:27 ` Graham Cobb 2021-09-23 9:40 ` Yuxuan Shui 1 sibling, 1 reply; 19+ messages in thread From: Graham Cobb @ 2021-09-23 9:27 UTC (permalink / raw) To: Yuxuan Shui, Qu Wenruo; +Cc: linux-btrfs On 23/09/2021 02:34, Yuxuan Shui wrote: > Hi, > > On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: >> >> >> >> On 2021/9/23 03:37, Yuxuan Shui wrote: >>> Hi, >>> >>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: >>> >>> mkfile o119493905-1537066-0 >>> rename o119493905-1537066-0 -> >>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h >>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include >>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h >>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h >>> source offset=0 offset=0 length=131072 >>> ERROR: failed to clone extents to >>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: >>> Invalid argument >>> >>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, >>> on the receiving end: >>> >>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h >>> Size: 145904 Blocks: 288 IO Block: 4096 regular file >>> >>> Looks to me the range of clone is within the boundary of the source >>> file. Not sure why this failed? >> >> The most common reason is, you have changed the parent subvolume from RO >> to RW, and modified the parent subvolume, then converted it back to RO. > > This is 100% not the case. I created these snapshots as RO right > before sending, and definitely haven't > changed them to RW ever. The problem isn't with the snapshots on the sending side, it is with the snapshots on the receiving side. Are you certain the snapshot on the receiving end has not been touched in any way (in particular, never been set to "RW" at any time)? Graham ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-23 9:27 ` Graham Cobb @ 2021-09-23 9:40 ` Yuxuan Shui 0 siblings, 0 replies; 19+ messages in thread From: Yuxuan Shui @ 2021-09-23 9:40 UTC (permalink / raw) To: Graham Cobb; +Cc: Qu Wenruo, linux-btrfs On Thu, Sep 23, 2021 at 10:27 AM Graham Cobb <g.btrfs@cobb.uk.net> wrote: > > On 23/09/2021 02:34, Yuxuan Shui wrote: > > Hi, > > > > On Thu, Sep 23, 2021 at 12:24 AM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote: > >> > >> > >> > >> On 2021/9/23 03:37, Yuxuan Shui wrote: > >>> Hi, > >>> > >>> The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > >>> > >>> mkfile o119493905-1537066-0 > >>> rename o119493905-1537066-0 -> > >>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>> utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > >>> clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > >>> - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>> source offset=0 offset=0 length=131072 > >>> ERROR: failed to clone extents to > >>> shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > >>> Invalid argument > >>> > >>> stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > >>> on the receiving end: > >>> > >>> File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > >>> Size: 145904 Blocks: 288 IO Block: 4096 regular file > >>> > >>> Looks to me the range of clone is within the boundary of the source > >>> file. Not sure why this failed? > >> > >> The most common reason is, you have changed the parent subvolume from RO > >> to RW, and modified the parent subvolume, then converted it back to RO. > > > > This is 100% not the case. I created these snapshots as RO right > > before sending, and definitely haven't > > changed them to RW ever. > > The problem isn't with the snapshots on the sending side, it is with the > snapshots on the receiving side. Are you certain the snapshot on the > receiving end has not been touched in any way (in particular, never been > set to "RW" at any time)? I can guarantee you that neither ends have ever been set to RW. > > Graham -- Regards Yuxuan Shui ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: btrfs receive fails with "failed to clone extents" 2021-09-22 19:37 btrfs receive fails with "failed to clone extents" Yuxuan Shui 2021-09-22 23:24 ` Qu Wenruo @ 2021-09-23 10:13 ` Filipe Manana 1 sibling, 0 replies; 19+ messages in thread From: Filipe Manana @ 2021-09-23 10:13 UTC (permalink / raw) To: Yuxuan Shui; +Cc: linux-btrfs On Wed, Sep 22, 2021 at 8:40 PM Yuxuan Shui <yshuiv7@gmail.com> wrote: > > Hi, > > The problem is as the title states. Relevant logs from `btrfs receive -vvv`: > > mkfile o119493905-1537066-0 > rename o119493905-1537066-0 -> > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > utimes shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include > clone shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h > - source=shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > source offset=0 offset=0 length=131072 > ERROR: failed to clone extents to > shui/programs/treeusage/target/release/build/zstd-sys-506c8effd111251c/out/include/zstd.h: > Invalid argument > > stat of shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h, > on the receiving end: > > File: /mnt/backup/home/backup-32/shui/.cargo/registry/src/github.com-1ecc6299db9ec823/zstd-sys-1.6.1+zstd.1.5.0/zstd/lib/zstd.h > Size: 145904 Blocks: 288 IO Block: 4096 regular file > > Looks to me the range of clone is within the boundary of the source > file. Not sure why this failed? > > Sending end has 5.14.6 and btrfs-progs 5.14, receiving end has 5.14.6 > and btrfs-progs 5.13.1 So, this may or may not be due to the case Darrel ran into recently: https://lore.kernel.org/linux-btrfs/CAOaVUnV3L6RpcqJ5gaqzNXWXK0VMkEVXCdihawH1PgS6TiMchQ@mail.gmail.com/ If you follow the thread, there's a list of steps and things useful for debugging there. Also, to really check if you have snapshots with a duplicated received_uuid, run the following on each filesystem (i.e. sending side and receiving side): btrfs subvolume list -puqR <mount point sending side> btrfs subvolume list -puqR <mount point receiving side> Finally, what's the exact command you are using to do the incremental send? Are you passing 1 or more clone roots (-c option), or just -p? Also, when you 'btrfs receive -vvv', what's the very first line? It should be something like this: receiving snapshot mysnap2 uuid=3208d767-fa97-3340-b5d5-33f64ba76458, ctransid=7 parent_uuid=d96591a4-7fbf-d44e-a8a8-c46fd1c7380d, parent_ctransid=6 The output of receive -vvv is not enough to take any conclusion yet. The operation will be invalid if we end up trying to clone from the same file and root, because source and destination offsets are the same. But if it's from a different root and file, or just from a different root but the same file, then it may or may not be invalid, depending on the source file size. One way to help figure that out is to use one of debug patches (for btrfs-progs) I pasted on that thread, which is also at https://pastebin.com/raw/9CbN9C0H Thanks. > > -- > > Regards > Yuxuan Shui -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2021-09-24 9:58 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-22 19:37 btrfs receive fails with "failed to clone extents" Yuxuan Shui
2021-09-22 23:24 ` Qu Wenruo
2021-09-23 1:34 ` Yuxuan Shui
2021-09-23 1:40 ` Yuxuan Shui
2021-09-23 2:32 ` Qu Wenruo
[not found] ` <CAGqt0zz+=nUYbNwLSPYwzYcNLgyxsWT22p+jwwFpAOcyAHAWgA@mail.gmail.com>
2021-09-23 9:27 ` Fwd: " Yuxuan Shui
[not found] ` <e83029f0-8583-91b9-47c8-a99d4b00a6ae@gmx.com>
2021-09-23 9:38 ` Yuxuan Shui
2021-09-23 9:43 ` Yuxuan Shui
2021-09-23 9:44 ` Qu Wenruo
2021-09-23 10:08 ` Yuxuan Shui
2021-09-23 10:19 ` Qu Wenruo
2021-09-23 10:46 ` Filipe Manana
2021-09-23 11:16 ` Qu Wenruo
2021-09-23 11:24 ` Filipe Manana
2021-09-24 9:47 ` Yuxuan Shui
2021-09-24 9:57 ` Qu Wenruo
2021-09-23 9:27 ` Graham Cobb
2021-09-23 9:40 ` Yuxuan Shui
2021-09-23 10:13 ` Filipe Manana
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.