* Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory @ 2020-01-13 12:41 Craig Andrews 2020-01-13 13:37 ` David Sterba 0 siblings, 1 reply; 14+ messages in thread From: Craig Andrews @ 2020-01-13 12:41 UTC (permalink / raw) To: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1083 bytes --] If I perform a btrfs send receive like so: sh -c btrfs send -p /mnt/everything/.snapshots/root.20191230 /mnt/everything/.snapshots/root.20191231 | btrfs receive /mnt/backup/.snapshots/ On Linux 5.4.0, the process completes successfully. Starting with Linux 5.5.0-rc1 up to the current 5.5 rc, 5.5.0-rc5, the result is the OOM killer being invoked which (among other process carnage) kills the btrfs processes stopping the backup. I'm using the same kernel config, same hardware, etc in the two tests. The system has 16 GB of RAM, CPU is an i5-6500, arch is amd64. /mnt/everything is a btrfs fs on luks (/dev/mapper/sda4) on a 1 TB SATA SSD (/dev/sda4). /mnt/backup is btfs fs on luks (/dev/mapper/backup) on a 2 TB external spinning rust HDD (/dev/sdb). # df -H /mnt/everything/ /mnt/backup Filesystem Size Used Avail Use% Mounted on /dev/mapper/sda4 949G 698G 247G 74% /mnt/everything /dev/mapper/backup 2.1T 1.1T 948G 53% /mnt/backup I've attached logs and my kernel config to https://bugzilla.kernel.org/show_bug.cgi?id=206031 Thank you, ~Craig [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-13 12:41 Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory Craig Andrews @ 2020-01-13 13:37 ` David Sterba 2020-01-13 14:48 ` Craig Andrews ` (2 more replies) 0 siblings, 3 replies; 14+ messages in thread From: David Sterba @ 2020-01-13 13:37 UTC (permalink / raw) To: Craig Andrews; +Cc: linux-btrfs On Mon, Jan 13, 2020 at 07:41:13AM -0500, Craig Andrews wrote: > If I perform a btrfs send receive like so: > > sh -c btrfs send -p /mnt/everything/.snapshots/root.20191230 > /mnt/everything/.snapshots/root.20191231 | btrfs receive > /mnt/backup/.snapshots/ > > On Linux 5.4.0, the process completes successfully. > > Starting with Linux 5.5.0-rc1 up to the current 5.5 rc, 5.5.0-rc5, the > result is the OOM killer being invoked which (among other process > carnage) kills the btrfs processes stopping the backup. As this is on the -rc1, it's possible that changes done in the MM subsystem could change the logic of OOM and that send now is able to trigger the OOM. There are 2 btrfs patches in send.c but they reduce amount of work, namely for heavily reflinked extents so the effects should be opposite. To find out where's the cause, is it possible that you build a kernel from the 5.4-based branch and run the send again? It's the same set of btrfs patches that gets merged to 5.5. From repository: git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git branch: for-5.5-rc4 If this can't be done we'll find another way to debug it. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-13 13:37 ` David Sterba @ 2020-01-13 14:48 ` Craig Andrews 2020-01-15 18:44 ` Craig Andrews 2020-01-21 2:18 ` Craig Andrews 2020-01-25 11:08 ` Stéphane Lesimple 2 siblings, 1 reply; 14+ messages in thread From: Craig Andrews @ 2020-01-13 14:48 UTC (permalink / raw) To: Craig Andrews; +Cc: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1407 bytes --] On 2020-01-13 08:37, David Sterba wrote: > On Mon, Jan 13, 2020 at 07:41:13AM -0500, Craig Andrews wrote: >> If I perform a btrfs send receive like so: >> >> sh -c btrfs send -p /mnt/everything/.snapshots/root.20191230 >> /mnt/everything/.snapshots/root.20191231 | btrfs receive >> /mnt/backup/.snapshots/ >> >> On Linux 5.4.0, the process completes successfully. >> >> Starting with Linux 5.5.0-rc1 up to the current 5.5 rc, 5.5.0-rc5, the >> result is the OOM killer being invoked which (among other process >> carnage) kills the btrfs processes stopping the backup. > > As this is on the -rc1, it's possible that changes done in the MM > subsystem could change the logic of OOM and that send now is able to > trigger the OOM. > > There are 2 btrfs patches in send.c but they reduce amount of work, > namely for heavily reflinked extents so the effects should be opposite. > > To find out where's the cause, is it possible that you build a kernel > from the 5.4-based branch and run the send again? It's the same set of > btrfs patches that gets merged to 5.5. > > From repository: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git > branch: for-5.5-rc4 > > If this can't be done we'll find another way to debug it. I've grabbed, compiled, and installed the kernel. I'll be able to tell you the results of running the test tomorrow at about this time. Thank you for help, ~Craig [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-13 14:48 ` Craig Andrews @ 2020-01-15 18:44 ` Craig Andrews 0 siblings, 0 replies; 14+ messages in thread From: Craig Andrews @ 2020-01-15 18:44 UTC (permalink / raw) To: Craig Andrews; +Cc: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1725 bytes --] On 2020-01-13 09:48, Craig Andrews wrote: > On 2020-01-13 08:37, David Sterba wrote: >> On Mon, Jan 13, 2020 at 07:41:13AM -0500, Craig Andrews wrote: >>> If I perform a btrfs send receive like so: >>> >>> sh -c btrfs send -p /mnt/everything/.snapshots/root.20191230 >>> /mnt/everything/.snapshots/root.20191231 | btrfs receive >>> /mnt/backup/.snapshots/ >>> >>> On Linux 5.4.0, the process completes successfully. >>> >>> Starting with Linux 5.5.0-rc1 up to the current 5.5 rc, 5.5.0-rc5, >>> the >>> result is the OOM killer being invoked which (among other process >>> carnage) kills the btrfs processes stopping the backup. >> >> As this is on the -rc1, it's possible that changes done in the MM >> subsystem could change the logic of OOM and that send now is able to >> trigger the OOM. >> >> There are 2 btrfs patches in send.c but they reduce amount of work, >> namely for heavily reflinked extents so the effects should be >> opposite. >> >> To find out where's the cause, is it possible that you build a kernel >> from the 5.4-based branch and run the send again? It's the same set of >> btrfs patches that gets merged to 5.5. >> >> From repository: >> git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git >> branch: for-5.5-rc4 >> >> If this can't be done we'll find another way to debug it. > > I've grabbed, compiled, and installed the kernel. I'll be able to tell > you the results of running the test tomorrow at about this time. > I couldn't get the kernel to boot so I need to spend more time on that. Is there any info I can get that would be helpful using my current kernels (either 5.4 or 5.5.0-rc5) while I work out why I can't boot the kernel you linked me to? Thanks, ~Craig [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-13 13:37 ` David Sterba 2020-01-13 14:48 ` Craig Andrews @ 2020-01-21 2:18 ` Craig Andrews 2020-01-25 11:08 ` Stéphane Lesimple 2 siblings, 0 replies; 14+ messages in thread From: Craig Andrews @ 2020-01-21 2:18 UTC (permalink / raw) To: Craig Andrews; +Cc: linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1821 bytes --] On 2020-01-13 08:37, David Sterba wrote: > On Mon, Jan 13, 2020 at 07:41:13AM -0500, Craig Andrews wrote: >> If I perform a btrfs send receive like so: >> >> sh -c btrfs send -p /mnt/everything/.snapshots/root.20191230 >> /mnt/everything/.snapshots/root.20191231 | btrfs receive >> /mnt/backup/.snapshots/ >> >> On Linux 5.4.0, the process completes successfully. >> >> Starting with Linux 5.5.0-rc1 up to the current 5.5 rc, 5.5.0-rc5, the >> result is the OOM killer being invoked which (among other process >> carnage) kills the btrfs processes stopping the backup. > > As this is on the -rc1, it's possible that changes done in the MM > subsystem could change the logic of OOM and that send now is able to > trigger the OOM. > > There are 2 btrfs patches in send.c but they reduce amount of work, > namely for heavily reflinked extents so the effects should be opposite. > > To find out where's the cause, is it possible that you build a kernel > from the 5.4-based branch and run the send again? It's the same set of > btrfs patches that gets merged to 5.5. > > From repository: > git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux.git > branch: for-5.5-rc4 > > If this can't be done we'll find another way to debug it. I finally was able to test with this kernel. With the kernel compiled from the branch "for-5.5-rc4" I do not get the out of memory error. However, I do this error: btrfs send -p /mnt/everything/.snapshots/root.20200119 /mnt/everything/.snapshots/root.20200120 | mbuffer -v 1 | btrfs receive /mnt/backup/.snapshots/ ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument I rebooted then immediately ran the same command with the 5.4 kernel and it completed successfully. So I guess it's nice that I didn't get an out of memory error... Thanks, ~Craig [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-13 13:37 ` David Sterba 2020-01-13 14:48 ` Craig Andrews 2020-01-21 2:18 ` Craig Andrews @ 2020-01-25 11:08 ` Stéphane Lesimple 2020-01-27 13:44 ` Filipe Manana 2020-01-27 20:08 ` Stéphane Lesimple 2 siblings, 2 replies; 14+ messages in thread From: Stéphane Lesimple @ 2020-01-25 11:08 UTC (permalink / raw) To: Craig Andrews; +Cc: linux-btrfs > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument If I may add another data point here, I'm also encountering this issue on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as btrfs is concerned, this is an rc7). On the first time, it happened after sending ~90 Gb worth of data, and aborted (as I didn't specify the -E option to btrfs send). Then, I retried with btrfs send -E 0, and it encountered the exact same error on the same file. # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ At subvol /tank/backups/.snaps/incoming/sendme/ At subvol sendme ERROR: failed to clone extents to retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument ERROR: failed to clone extents to retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument The send/receive is still going on for now, currently around the ~200 Gb mark. Bees is running on this FS (but I stopped it before doing the send/receive). I can test patches if needed. -- Stéphane. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-25 11:08 ` Stéphane Lesimple @ 2020-01-27 13:44 ` Filipe Manana 2020-01-28 14:46 ` Craig Andrews 2020-01-27 20:08 ` Stéphane Lesimple 1 sibling, 1 reply; 14+ messages in thread From: Filipe Manana @ 2020-01-27 13:44 UTC (permalink / raw) To: Stéphane Lesimple; +Cc: Craig Andrews, linux-btrfs On Sat, Jan 25, 2020 at 11:18 AM Stéphane Lesimple <stephane_btrfs2@lesimple.fr> wrote: > > > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument > > If I may add another data point here, I'm also encountering this issue on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as btrfs is concerned, this is an rc7). > > On the first time, it happened after sending ~90 Gb worth of data, and aborted (as I didn't specify the -E option to btrfs send). Then, I retried with btrfs send -E 0, and it encountered the exact same error on the same file. > > # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ > At subvol /tank/backups/.snaps/incoming/sendme/ > At subvol sendme > ERROR: failed to clone extents to retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > ERROR: failed to clone extents to retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument This is probably the same case for which I sent a fix last week: https://patchwork.kernel.org/patch/11350129/ Thanks. > > The send/receive is still going on for now, currently around the ~200 Gb mark. > > Bees is running on this FS (but I stopped it before doing the send/receive). > > I can test patches if needed. > > -- > Stéphane. -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-27 13:44 ` Filipe Manana @ 2020-01-28 14:46 ` Craig Andrews 2020-01-28 15:02 ` Filipe Manana 0 siblings, 1 reply; 14+ messages in thread From: Craig Andrews @ 2020-01-28 14:46 UTC (permalink / raw) To: fdmanana; +Cc: Stéphane Lesimple, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1627 bytes --] On 2020-01-27 08:44, Filipe Manana wrote: > On Sat, Jan 25, 2020 at 11:18 AM Stéphane Lesimple > <stephane_btrfs2@lesimple.fr> wrote: >> >> > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument >> >> If I may add another data point here, I'm also encountering this issue >> on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as >> btrfs is concerned, this is an rc7). >> >> On the first time, it happened after sending ~90 Gb worth of data, and >> aborted (as I didn't specify the -E option to btrfs send). Then, I >> retried with btrfs send -E 0, and it encountered the exact same error >> on the same file. >> >> # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv >> 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ >> At subvol /tank/backups/.snaps/incoming/sendme/ >> At subvol sendme >> ERROR: failed to clone extents to >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument >> ERROR: failed to clone extents to >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > > This is probably the same case for which I sent a fix last week: > > https://patchwork.kernel.org/patch/11350129/ > > Thanks. I applied that patch to 5.5.0-rc7 and that solved the problem. I can now do backups (which is a send/receive) successfully. > >> >> The send/receive is still going on for now, currently around the ~200 >> Gb mark. >> >> Bees is running on this FS (but I stopped it before doing the >> send/receive). >> >> I can test patches if needed. >> >> -- >> Stéphane. The patch appears to have fixed my problems - thank you! ~Craig [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-28 14:46 ` Craig Andrews @ 2020-01-28 15:02 ` Filipe Manana 2020-01-28 15:05 ` Craig Andrews 0 siblings, 1 reply; 14+ messages in thread From: Filipe Manana @ 2020-01-28 15:02 UTC (permalink / raw) To: Craig Andrews; +Cc: Stéphane Lesimple, linux-btrfs On Tue, Jan 28, 2020 at 2:46 PM Craig Andrews <candrews@integralblue.com> wrote: > > On 2020-01-27 08:44, Filipe Manana wrote: > > On Sat, Jan 25, 2020 at 11:18 AM Stéphane Lesimple > > <stephane_btrfs2@lesimple.fr> wrote: > >> > >> > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument > >> > >> If I may add another data point here, I'm also encountering this issue > >> on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as > >> btrfs is concerned, this is an rc7). > >> > >> On the first time, it happened after sending ~90 Gb worth of data, and > >> aborted (as I didn't specify the -E option to btrfs send). Then, I > >> retried with btrfs send -E 0, and it encountered the exact same error > >> on the same file. > >> > >> # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv > >> 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ > >> At subvol /tank/backups/.snaps/incoming/sendme/ > >> At subvol sendme > >> ERROR: failed to clone extents to > >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > >> ERROR: failed to clone extents to > >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > > > > This is probably the same case for which I sent a fix last week: > > > > https://patchwork.kernel.org/patch/11350129/ > > > > Thanks. > > I applied that patch to 5.5.0-rc7 and that solved the problem. I can now > do backups (which is a send/receive) successfully. > > > > >> > >> The send/receive is still going on for now, currently around the ~200 > >> Gb mark. > >> > >> Bees is running on this FS (but I stopped it before doing the > >> send/receive). > >> > >> I can test patches if needed. > >> > >> -- > >> Stéphane. > > The patch appears to have fixed my problems - thank you! Great! Can you reply to the patch's thread with a: Tested-by: Your Name <email@foo.bar> ? Or I can reply to it myself with that if you agree. Thanks! > ~Craig -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-28 15:02 ` Filipe Manana @ 2020-01-28 15:05 ` Craig Andrews 2020-01-28 15:08 ` Filipe Manana 2020-01-28 17:31 ` David Sterba 0 siblings, 2 replies; 14+ messages in thread From: Craig Andrews @ 2020-01-28 15:05 UTC (permalink / raw) To: fdmanana; +Cc: Stéphane Lesimple, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2183 bytes --] On 2020-01-28 10:02, Filipe Manana wrote: > On Tue, Jan 28, 2020 at 2:46 PM Craig Andrews > <candrews@integralblue.com> wrote: >> >> On 2020-01-27 08:44, Filipe Manana wrote: >> > On Sat, Jan 25, 2020 at 11:18 AM Stéphane Lesimple >> > <stephane_btrfs2@lesimple.fr> wrote: >> >> >> >> > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument >> >> >> >> If I may add another data point here, I'm also encountering this issue >> >> on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as >> >> btrfs is concerned, this is an rc7). >> >> >> >> On the first time, it happened after sending ~90 Gb worth of data, and >> >> aborted (as I didn't specify the -E option to btrfs send). Then, I >> >> retried with btrfs send -E 0, and it encountered the exact same error >> >> on the same file. >> >> >> >> # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv >> >> 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ >> >> At subvol /tank/backups/.snaps/incoming/sendme/ >> >> At subvol sendme >> >> ERROR: failed to clone extents to >> >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument >> >> ERROR: failed to clone extents to >> >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument >> > >> > This is probably the same case for which I sent a fix last week: >> > >> > https://patchwork.kernel.org/patch/11350129/ >> > >> > Thanks. >> >> I applied that patch to 5.5.0-rc7 and that solved the problem. I can >> now >> do backups (which is a send/receive) successfully. >> >> > >> >> >> >> The send/receive is still going on for now, currently around the ~200 >> >> Gb mark. >> >> >> >> Bees is running on this FS (but I stopped it before doing the >> >> send/receive). >> >> >> >> I can test patches if needed. >> >> >> >> -- >> >> Stéphane. >> >> The patch appears to have fixed my problems - thank you! > > Great! > > Can you reply to the patch's thread with a: > > Tested-by: Your Name <email@foo.bar> > > ? > > Or I can reply to it myself with that if you agree. > > Thanks! > >> ~Craig Can you please send the reply? Tested-by: Craig Andrews <candrews@integralblue.com> And again, thank you! ~Craig [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-28 15:05 ` Craig Andrews @ 2020-01-28 15:08 ` Filipe Manana 2020-01-28 17:33 ` Filipe Manana 2020-01-28 17:31 ` David Sterba 1 sibling, 1 reply; 14+ messages in thread From: Filipe Manana @ 2020-01-28 15:08 UTC (permalink / raw) To: Craig Andrews; +Cc: Stéphane Lesimple, linux-btrfs On Tue, Jan 28, 2020 at 3:05 PM Craig Andrews <candrews@integralblue.com> wrote: > > On 2020-01-28 10:02, Filipe Manana wrote: > > On Tue, Jan 28, 2020 at 2:46 PM Craig Andrews > > <candrews@integralblue.com> wrote: > >> > >> On 2020-01-27 08:44, Filipe Manana wrote: > >> > On Sat, Jan 25, 2020 at 11:18 AM Stéphane Lesimple > >> > <stephane_btrfs2@lesimple.fr> wrote: > >> >> > >> >> > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument > >> >> > >> >> If I may add another data point here, I'm also encountering this issue > >> >> on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as > >> >> btrfs is concerned, this is an rc7). > >> >> > >> >> On the first time, it happened after sending ~90 Gb worth of data, and > >> >> aborted (as I didn't specify the -E option to btrfs send). Then, I > >> >> retried with btrfs send -E 0, and it encountered the exact same error > >> >> on the same file. > >> >> > >> >> # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv > >> >> 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ > >> >> At subvol /tank/backups/.snaps/incoming/sendme/ > >> >> At subvol sendme > >> >> ERROR: failed to clone extents to > >> >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > >> >> ERROR: failed to clone extents to > >> >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > >> > > >> > This is probably the same case for which I sent a fix last week: > >> > > >> > https://patchwork.kernel.org/patch/11350129/ > >> > > >> > Thanks. > >> > >> I applied that patch to 5.5.0-rc7 and that solved the problem. I can > >> now > >> do backups (which is a send/receive) successfully. > >> > >> > > >> >> > >> >> The send/receive is still going on for now, currently around the ~200 > >> >> Gb mark. > >> >> > >> >> Bees is running on this FS (but I stopped it before doing the > >> >> send/receive). > >> >> > >> >> I can test patches if needed. > >> >> > >> >> -- > >> >> Stéphane. > >> > >> The patch appears to have fixed my problems - thank you! > > > > Great! > > > > Can you reply to the patch's thread with a: > > > > Tested-by: Your Name <email@foo.bar> > > > > ? > > > > Or I can reply to it myself with that if you agree. > > > > Thanks! > > > >> ~Craig > > > Can you please send the reply? > Tested-by: Craig Andrews <candrews@integralblue.com> No problem, done. Thanks. > > And again, thank you! > ~Craig -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-28 15:08 ` Filipe Manana @ 2020-01-28 17:33 ` Filipe Manana 0 siblings, 0 replies; 14+ messages in thread From: Filipe Manana @ 2020-01-28 17:33 UTC (permalink / raw) To: Craig Andrews; +Cc: Stéphane Lesimple, linux-btrfs On Tue, Jan 28, 2020 at 3:08 PM Filipe Manana <fdmanana@gmail.com> wrote: > > On Tue, Jan 28, 2020 at 3:05 PM Craig Andrews <candrews@integralblue.com> wrote: > > > > On 2020-01-28 10:02, Filipe Manana wrote: > > > On Tue, Jan 28, 2020 at 2:46 PM Craig Andrews > > > <candrews@integralblue.com> wrote: > > >> > > >> On 2020-01-27 08:44, Filipe Manana wrote: > > >> > On Sat, Jan 25, 2020 at 11:18 AM Stéphane Lesimple > > >> > <stephane_btrfs2@lesimple.fr> wrote: > > >> >> > > >> >> > ERROR: failed to clone extents to var/amavis/db/__db.001: Invalid argument > > >> >> > > >> >> If I may add another data point here, I'm also encountering this issue > > >> >> on a 5.5.0-rc6, with the btrfs-rc7 patches applied to it (so as far as > > >> >> btrfs is concerned, this is an rc7). > > >> >> > > >> >> On the first time, it happened after sending ~90 Gb worth of data, and > > >> >> aborted (as I didn't specify the -E option to btrfs send). Then, I > > >> >> retried with btrfs send -E 0, and it encountered the exact same error > > >> >> on the same file. > > >> >> > > >> >> # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv > > >> >> 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ > > >> >> At subvol /tank/backups/.snaps/incoming/sendme/ > > >> >> At subvol sendme > > >> >> ERROR: failed to clone extents to > > >> >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > > >> >> ERROR: failed to clone extents to > > >> >> retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid argument > > >> > > > >> > This is probably the same case for which I sent a fix last week: > > >> > > > >> > https://patchwork.kernel.org/patch/11350129/ > > >> > > > >> > Thanks. > > >> > > >> I applied that patch to 5.5.0-rc7 and that solved the problem. I can > > >> now > > >> do backups (which is a send/receive) successfully. > > >> > > >> > > > >> >> > > >> >> The send/receive is still going on for now, currently around the ~200 > > >> >> Gb mark. > > >> >> > > >> >> Bees is running on this FS (but I stopped it before doing the > > >> >> send/receive). > > >> >> > > >> >> I can test patches if needed. > > >> >> > > >> >> -- > > >> >> Stéphane. > > >> > > >> The patch appears to have fixed my problems - thank you! > > > > > > Great! > > > > > > Can you reply to the patch's thread with a: > > > > > > Tested-by: Your Name <email@foo.bar> > > > > > > ? > > > > > > Or I can reply to it myself with that if you agree. > > > > > > Thanks! > > > > > >> ~Craig > > > > > > Can you please send the reply? > > Tested-by: Craig Andrews <candrews@integralblue.com> > > No problem, done. Thanks. Actually I just realized one chunk of the patch can cause problems with fallocated extents beyond a file's size and a hole that ends right at the file's size (unlikely to be a common scenario anyway). I also don't think it's needed. For my tests at least, removing it doesn't cause any new problems and solves the problem with the invalid clone operations. Can you try it for your use case too? To be clear, this is the new version of the patch: https://pastebin.com/raw/vewZiG9J (A shorter version of the original) Thanks! > > > > > And again, thank you! > > ~Craig > > > > -- > Filipe David Manana, > > “Whether you think you can, or you think you can't — you're right.” -- Filipe David Manana, “Whether you think you can, or you think you can't — you're right.” ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-28 15:05 ` Craig Andrews 2020-01-28 15:08 ` Filipe Manana @ 2020-01-28 17:31 ` David Sterba 1 sibling, 0 replies; 14+ messages in thread From: David Sterba @ 2020-01-28 17:31 UTC (permalink / raw) To: Craig Andrews; +Cc: fdmanana, Stéphane Lesimple, linux-btrfs On Tue, Jan 28, 2020 at 10:05:37AM -0500, Craig Andrews wrote: > Can you please send the reply? > Tested-by: Craig Andrews <candrews@integralblue.com> Thanks for the report and testing! As the patch is a fix it will go to mainline soon and then it'll propagate to stable trees, 5.4 and 5.5. ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory 2020-01-25 11:08 ` Stéphane Lesimple 2020-01-27 13:44 ` Filipe Manana @ 2020-01-27 20:08 ` Stéphane Lesimple 1 sibling, 0 replies; 14+ messages in thread From: Stéphane Lesimple @ 2020-01-27 20:08 UTC (permalink / raw) To: fdmanana; +Cc: Craig Andrews, linux-btrfs >> # btrfs send -v /tank/backups/.snaps/incoming/sendme/ | pv 2>/dev/pts/23 | btrfs rec -E 0 /newfs/ >> At subvol /tank/backups/.snaps/incoming/sendme/ >> At subvol sendme >> ERROR: failed to clone extents to retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid >> argument >> ERROR: failed to clone extents to retroarch/x86_64/cores/mednafen_saturn_libretro.dll: Invalid >> argument > > This is probably the same case for which I sent a fix last week: > > https://patchwork.kernel.org/patch/11350129 This seems very likely, as there is indeed a hole in the middle of the file: 0: ram 80000 disk(0) 25871fb42000:00000000-00080000 disk_sz 80000 80000: ram 80000 disk(0) 258720475000:00000000-00080000 disk_sz 80000 100000: ram 80000 disk(0) 2587215d4000:00000000-00080000 disk_sz 80000 180000: ram 80000 disk(0) 258721654000:00000000-00080000 disk_sz 80000 200000: ram 80000 disk(0) 25872177c000:00000000-00080000 disk_sz 80000 280000: ram 80000 disk(0) 258722383000:00000000-00080000 disk_sz 80000 300000: ram 80000 disk(0) 258722544000:00000000-00080000 disk_sz 80000 380000: ram 80000 disk(0) 25871c3a8000:00000000-00080000 disk_sz 80000 400000: ram 80000 disk(0) 25871c428000:00000000-00080000 disk_sz 80000 480000: ram 80000 disk(0) 25871c548000:00000000-00080000 disk_sz 80000 500000: ram 80000 disk(0) 25871cf02000:00000000-00080000 disk_sz 80000 580000: ram 80000 disk(0) 25871cf82000:00000000-00080000 disk_sz 80000 600000: ram 80000 disk(0) 25871d41f000:00000000-00080000 disk_sz 80000 680000: ram 80000 disk(0) 25871d833000:00000000-00080000 disk_sz 80000 700000: ram b000 disk(0) b9e86aeb000:00000000-0000b000 disk_sz b000 70b000: ram 11000 disk(0) 0:00000000-00011000 disk_sz 0 -- hole 71c000: ram 80000 disk(0) 25871eb75000:00000000-00080000 disk_sz 80000 79c000: ram 1000 disk(0) a00b6523000:00000000-00001000 disk_sz 1000 79d000: ram 1000 disk(0) a00b6cf5000:00000000-00001000 disk_sz 1000 79e000: ram 1000 disk(0) a00b6cf5000:00000000-00001000 disk_sz 1000 79f000: ram 1000 disk(0) a00b6cf5000:00000000-00001000 disk_sz 1000 7a0000: ram 1000 disk(0) a00b6d42000:00000000-00001000 disk_sz 1000 7a1000: ram 1000 disk(0) a9573bfa000:00000000-00001000 disk_sz 6000 7a2000: ram 1000 disk(0) a9573bfa000:00000000-00001000 disk_sz 6000 7a3000: ram 6000 disk(0) a9573bfa000:00000000-00006000 disk_sz 6000 7a9000: ram 1000 disk(0) af4f6240000:00000000-00001000 disk_sz 2d000 7aa000: ram 1000 disk(0) af4f6240000:00000000-00001000 disk_sz 2d000 7ab000: ram 2d000 disk(0) af4f6240000:00000000-0002d000 disk_sz 2d000 7d8000: ram 1000 disk(0) 1c8d313ba000:0001d000-0001e000 disk_sz 80000 7d9000: ram 43000 disk(0) a983253d000:00000000-00043000 disk_sz 43000 81c000: ram 80000 disk(0) 258721f18000:00000000-00080000 disk_sz 80000 89c000: ram 80000 disk(0) 258721f98000:00000000-00080000 disk_sz 80000 91c000: ram 80000 disk(0) 258722018000:00000000-00080000 disk_sz 80000 99c000: ram 80000 disk(0) 2587229a2000:00000000-00080000 disk_sz 80000 a1c000: ram d000 disk(0) bfa6df44000:00000000-0000d000 disk_sz e000 a29000: ram 1000 disk(0) 130963b99000:00a29000-00a2a000 disk_sz a2a000 file: mednafen_saturn_libretro.dll extents 29 disk size 21741568 logical size 10653836 ratio 0.49 Thanks! -- Stéphane. ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2020-01-28 17:33 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-01-13 12:41 Regression in Linux 5.5.0-rc[1-5]: btrfs send/receive out of memory Craig Andrews 2020-01-13 13:37 ` David Sterba 2020-01-13 14:48 ` Craig Andrews 2020-01-15 18:44 ` Craig Andrews 2020-01-21 2:18 ` Craig Andrews 2020-01-25 11:08 ` Stéphane Lesimple 2020-01-27 13:44 ` Filipe Manana 2020-01-28 14:46 ` Craig Andrews 2020-01-28 15:02 ` Filipe Manana 2020-01-28 15:05 ` Craig Andrews 2020-01-28 15:08 ` Filipe Manana 2020-01-28 17:33 ` Filipe Manana 2020-01-28 17:31 ` David Sterba 2020-01-27 20:08 ` Stéphane Lesimple
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.