Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Hello again,
I'm not familiar with mailing list. Should I expect an answer sooner or later ?
As I need to get back on track as soon as possible, I would like to know if it's too complicated to get an answer quickly from you.
I don't want to be rude, I just want to know if I should wait long enough for an answer that might save my day and my data. Or I'm doomed and I should have wipe my drive already ?

I'll take any answer :)
Thank you

Nouts

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, April 28, 2020 11:26 AM, Nouts <nouts@protonmail.com> wrote:

> Hello,
>
> I am having issue with a RAID1 btrfs pool "failed to read block groups". I was advised to send information to this mailing list, as someone might be interested in debug logs and might also help solve my issue.
>
> I have a 3 drive RAID1 pool (2x3TB + 1x6TB), mounted as /home.
>
> My system became really slow while doing nothing, and after a reboot my /home pool can't mount.This is the error I got :
>
> [ 4645.402880] BTRFS info (device sdb): disk space caching is enabled
> [ 4645.405687] BTRFS info (device sdb): has skinny extents
> [ 4645.451484] BTRFS error (device sdb): failed to read block groups: -117
> [ 4645.472062] BTRFS error (device sdb): open_ctree failed
> mount: wrong fs type, bad option, bad superblock on /dev/sdb,missing codepage or helper program, or other error
> In some cases useful info is found in syslog - trydmesg | tail or so.
>
> I attached you the smartctl result from the day before and the last scrub report I got from a month ago. From my understanding, it was ok.
> I use hardlink (on the same partition/pool) and I deleted some data just the day before. I suspect my daily scrub routine triggered something that night and next day /home was gone.
>
> I can't scrub anymore as it's not mounted. Mounting with usebackuproot or degraded or ro produce the same error.
> I tried "btrfs check /dev/sda" :
> checking extents
> leaf parent key incorrect 5909107507200
> bad block 5909107507200
> Errors found in extent allocation tree or chunk allocation
> Checking filesystem on /dev/sda
> UUID: 3720251f-ef92-4e21-bad0-eae1c97cff03
>
> Then "btrfs rescue super-recover /dev/sda" :
> All supers are valid, no need to recover
>
> Then "btrfs rescue zero-log /dev/sda", which produced a weird stacktrace...
> Unable to find block group for 0
> extent-tree.c:289: find_search_start: Assertion '1' failed.
> btrfs[0x43e418]
> btrfs(btrfs_reserve_extent+0x5c9)[0x4425df]
> btrfs(btrfs_alloc_free_block+0x63[0x44297c]
> btrfs(__btrfs_cow_block+0xfc[0x436636]
> btrfs(btrfs_cow_block+0x8b)[0x436bd8]
> btrfs[0x43ad82]
> btrfs(btrfs_commit_transaction+0xb8)[0x43c5dc]
> btrfs[0x42c0d4]btrfs(main+0x12f)[0x40a341]/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f1462d712e1]
> btrfs(_start+0x2a)[0x40a37a]
> Clearing log on /dev/sda, previous log_root 0, level 0
>
> Finally I tried "btrfs rescue chunk-recover /dev/sda", which run on all 3 drives at the same time during 8+ hours...
> It asks to rebuild some metadata tree, which I accepted (I did not saved the full output sorry) and it ended with the same stacktrace as above.
>
> The only command left is "btrfs check --repair" but I afraid it might do more bad than good.
>
> I'm running Debian 9 (still, because of some dependencies). My kernel is already backported : 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux
> btrfs version : v4.7.3
> I originally posted on reddit : https://www.reddit.com/r/btrfs/comments/g99v4v/nas_raid1_not_mounting_failed_to_read_block_groups/
>
> Let me know if you need more information.
>
> Nouts

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Hugo Mills]

On Thu, Apr 30, 2020 at 11:41:11AM +0000, Nouts wrote:
> Hello again,
> I'm not familiar with mailing list. Should I expect an answer sooner or later ?

   Your original mail didn't get through to the list, as far as I can
see. I'd guess the attachment was too large, and the mail server
swallowed it.

   Hugo.

> As I need to get back on track as soon as possible, I would like to know if it's too complicated to get an answer quickly from you.
> I don't want to be rude, I just want to know if I should wait long enough for an answer that might save my day and my data. Or I'm doomed and I should have wipe my drive already ?
> 
> I'll take any answer :)
> Thank you
> 
> Nouts
> 
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, April 28, 2020 11:26 AM, Nouts <nouts@protonmail.com> wrote:
> 
> > Hello,
> >
> > I am having issue with a RAID1 btrfs pool "failed to read block groups". I was advised to send information to this mailing list, as someone might be interested in debug logs and might also help solve my issue.
> >
> > I have a 3 drive RAID1 pool (2x3TB + 1x6TB), mounted as /home.
> >
> > My system became really slow while doing nothing, and after a reboot my /home pool can't mount.This is the error I got :
> >
> > [ 4645.402880] BTRFS info (device sdb): disk space caching is enabled
> > [ 4645.405687] BTRFS info (device sdb): has skinny extents
> > [ 4645.451484] BTRFS error (device sdb): failed to read block groups: -117
> > [ 4645.472062] BTRFS error (device sdb): open_ctree failed
> > mount: wrong fs type, bad option, bad superblock on /dev/sdb,missing codepage or helper program, or other error
> > In some cases useful info is found in syslog - trydmesg | tail or so.
> >
> > I attached you the smartctl result from the day before and the last scrub report I got from a month ago. From my understanding, it was ok.
> > I use hardlink (on the same partition/pool) and I deleted some data just the day before. I suspect my daily scrub routine triggered something that night and next day /home was gone.
> >
> > I can't scrub anymore as it's not mounted. Mounting with usebackuproot or degraded or ro produce the same error.
> > I tried "btrfs check /dev/sda" :
> > checking extents
> > leaf parent key incorrect 5909107507200
> > bad block 5909107507200
> > Errors found in extent allocation tree or chunk allocation
> > Checking filesystem on /dev/sda
> > UUID: 3720251f-ef92-4e21-bad0-eae1c97cff03
> >
> > Then "btrfs rescue super-recover /dev/sda" :
> > All supers are valid, no need to recover
> >
> > Then "btrfs rescue zero-log /dev/sda", which produced a weird stacktrace...
> > Unable to find block group for 0
> > extent-tree.c:289: find_search_start: Assertion '1' failed.
> > btrfs[0x43e418]
> > btrfs(btrfs_reserve_extent+0x5c9)[0x4425df]
> > btrfs(btrfs_alloc_free_block+0x63[0x44297c]
> > btrfs(__btrfs_cow_block+0xfc[0x436636]
> > btrfs(btrfs_cow_block+0x8b)[0x436bd8]
> > btrfs[0x43ad82]
> > btrfs(btrfs_commit_transaction+0xb8)[0x43c5dc]
> > btrfs[0x42c0d4]btrfs(main+0x12f)[0x40a341]/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f1462d712e1]
> > btrfs(_start+0x2a)[0x40a37a]
> > Clearing log on /dev/sda, previous log_root 0, level 0
> >
> > Finally I tried "btrfs rescue chunk-recover /dev/sda", which run on all 3 drives at the same time during 8+ hours...
> > It asks to rebuild some metadata tree, which I accepted (I did not saved the full output sorry) and it ended with the same stacktrace as above.
> >
> > The only command left is "btrfs check --repair" but I afraid it might do more bad than good.
> >
> > I'm running Debian 9 (still, because of some dependencies). My kernel is already backported : 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux
> > btrfs version : v4.7.3
> > I originally posted on reddit : https://www.reddit.com/r/btrfs/comments/g99v4v/nas_raid1_not_mounting_failed_to_read_block_groups/
> >
> > Let me know if you need more information.
> >
> > Nouts
> 
> 

-- 
Hugo Mills             | To an Englishman, 100 miles is a long way; to an
hugo@... carfax.org.uk | American, 100 years is a long time.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                                        Earle Hitchner

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Arf ! Thanks for letting me know.
Here is a download link from my previous attachment
https://drop.tigwali.fr/r/PiTaYrbxwL#TXjCkEo4nx1GZkZV4tUlJeajpMi9GHa+hB60IOV+BqM=

I'll wait a bit more then, if you can tell what happened to my drive :)



‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, April 30, 2020 3:46 PM, Hugo Mills <hugo@carfax.org.uk> wrote:

> On Thu, Apr 30, 2020 at 11:41:11AM +0000, Nouts wrote:
>
> > Hello again,
> > I'm not familiar with mailing list. Should I expect an answer sooner or later ?
>
> Your original mail didn't get through to the list, as far as I can
> see. I'd guess the attachment was too large, and the mail server
> swallowed it.
>
> Hugo.
>
> > As I need to get back on track as soon as possible, I would like to know if it's too complicated to get an answer quickly from you.
> > I don't want to be rude, I just want to know if I should wait long enough for an answer that might save my day and my data. Or I'm doomed and I should have wipe my drive already ?
> > I'll take any answer :)
> > Thank you
> > Nouts
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Tuesday, April 28, 2020 11:26 AM, Nouts nouts@protonmail.com wrote:
> >
> > > Hello,
> > > I am having issue with a RAID1 btrfs pool "failed to read block groups". I was advised to send information to this mailing list, as someone might be interested in debug logs and might also help solve my issue.
> > > I have a 3 drive RAID1 pool (2x3TB + 1x6TB), mounted as /home.
> > > My system became really slow while doing nothing, and after a reboot my /home pool can't mount.This is the error I got :
> > > [ 4645.402880] BTRFS info (device sdb): disk space caching is enabled
> > > [ 4645.405687] BTRFS info (device sdb): has skinny extents
> > > [ 4645.451484] BTRFS error (device sdb): failed to read block groups: -117
> > > [ 4645.472062] BTRFS error (device sdb): open_ctree failed
> > > mount: wrong fs type, bad option, bad superblock on /dev/sdb,missing codepage or helper program, or other error
> > > In some cases useful info is found in syslog - trydmesg | tail or so.
> > > I attached you the smartctl result from the day before and the last scrub report I got from a month ago. From my understanding, it was ok.
> > > I use hardlink (on the same partition/pool) and I deleted some data just the day before. I suspect my daily scrub routine triggered something that night and next day /home was gone.
> > > I can't scrub anymore as it's not mounted. Mounting with usebackuproot or degraded or ro produce the same error.
> > > I tried "btrfs check /dev/sda" :
> > > checking extents
> > > leaf parent key incorrect 5909107507200
> > > bad block 5909107507200
> > > Errors found in extent allocation tree or chunk allocation
> > > Checking filesystem on /dev/sda
> > > UUID: 3720251f-ef92-4e21-bad0-eae1c97cff03
> > > Then "btrfs rescue super-recover /dev/sda" :
> > > All supers are valid, no need to recover
> > > Then "btrfs rescue zero-log /dev/sda", which produced a weird stacktrace...
> > > Unable to find block group for 0
> > > extent-tree.c:289: find_search_start: Assertion '1' failed.
> > > btrfs[0x43e418]
> > > btrfs(btrfs_reserve_extent+0x5c9)[0x4425df]
> > > btrfs(btrfs_alloc_free_block+0x63[0x44297c]
> > > btrfs(__btrfs_cow_block+0xfc[0x436636]
> > > btrfs(btrfs_cow_block+0x8b)[0x436bd8]
> > > btrfs[0x43ad82]
> > > btrfs(btrfs_commit_transaction+0xb8)[0x43c5dc]
> > > btrfs[0x42c0d4]btrfs(main+0x12f)[0x40a341]/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f1462d712e1]
> > > btrfs(_start+0x2a)[0x40a37a]
> > > Clearing log on /dev/sda, previous log_root 0, level 0
> > > Finally I tried "btrfs rescue chunk-recover /dev/sda", which run on all 3 drives at the same time during 8+ hours...
> > > It asks to rebuild some metadata tree, which I accepted (I did not saved the full output sorry) and it ended with the same stacktrace as above.
> > > The only command left is "btrfs check --repair" but I afraid it might do more bad than good.
> > > I'm running Debian 9 (still, because of some dependencies). My kernel is already backported : 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux
> > > btrfs version : v4.7.3
> > > I originally posted on reddit : https://www.reddit.com/r/btrfs/comments/g99v4v/nas_raid1_not_mounting_failed_to_read_block_groups/
> > > Let me know if you need more information.
> > > Nouts
>
> --
>
> Hugo Mills | To an Englishman, 100 miles is a long way; to an
> hugo@... carfax.org.uk | American, 100 years is a long time.
> http://carfax.org.uk/ |
> PGP: E2AB1DE4 | Earle Hitchner

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Edit : Sorry, Here is a working link
https://drop.tigwali.fr/r/r83Y1PBagL#1hveg/8cMAaXGQcosveNOs4XVUvTNPn1PX+c8S3xG2U=



‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, April 30, 2020 4:04 PM, Nouts <nouts@protonmail.com> wrote:

> Arf ! Thanks for letting me know.
> Here is a download link from my previous attachment
> https://drop.tigwali.fr/r/PiTaYrbxwL#TXjCkEo4nx1GZkZV4tUlJeajpMi9GHa+hB60IOV+BqM=
>
> I'll wait a bit more then, if you can tell what happened to my drive :)
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, April 30, 2020 3:46 PM, Hugo Mills hugo@carfax.org.uk wrote:
>
> > On Thu, Apr 30, 2020 at 11:41:11AM +0000, Nouts wrote:
> >
> > > Hello again,
> > > I'm not familiar with mailing list. Should I expect an answer sooner or later ?
> >
> > Your original mail didn't get through to the list, as far as I can
> > see. I'd guess the attachment was too large, and the mail server
> > swallowed it.
> > Hugo.
> >
> > > As I need to get back on track as soon as possible, I would like to know if it's too complicated to get an answer quickly from you.
> > > I don't want to be rude, I just want to know if I should wait long enough for an answer that might save my day and my data. Or I'm doomed and I should have wipe my drive already ?
> > > I'll take any answer :)
> > > Thank you
> > > Nouts
> > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > On Tuesday, April 28, 2020 11:26 AM, Nouts nouts@protonmail.com wrote:
> > >
> > > > Hello,
> > > > I am having issue with a RAID1 btrfs pool "failed to read block groups". I was advised to send information to this mailing list, as someone might be interested in debug logs and might also help solve my issue.
> > > > I have a 3 drive RAID1 pool (2x3TB + 1x6TB), mounted as /home.
> > > > My system became really slow while doing nothing, and after a reboot my /home pool can't mount.This is the error I got :
> > > > [ 4645.402880] BTRFS info (device sdb): disk space caching is enabled
> > > > [ 4645.405687] BTRFS info (device sdb): has skinny extents
> > > > [ 4645.451484] BTRFS error (device sdb): failed to read block groups: -117
> > > > [ 4645.472062] BTRFS error (device sdb): open_ctree failed
> > > > mount: wrong fs type, bad option, bad superblock on /dev/sdb,missing codepage or helper program, or other error
> > > > In some cases useful info is found in syslog - trydmesg | tail or so.
> > > > I attached you the smartctl result from the day before and the last scrub report I got from a month ago. From my understanding, it was ok.
> > > > I use hardlink (on the same partition/pool) and I deleted some data just the day before. I suspect my daily scrub routine triggered something that night and next day /home was gone.
> > > > I can't scrub anymore as it's not mounted. Mounting with usebackuproot or degraded or ro produce the same error.
> > > > I tried "btrfs check /dev/sda" :
> > > > checking extents
> > > > leaf parent key incorrect 5909107507200
> > > > bad block 5909107507200
> > > > Errors found in extent allocation tree or chunk allocation
> > > > Checking filesystem on /dev/sda
> > > > UUID: 3720251f-ef92-4e21-bad0-eae1c97cff03
> > > > Then "btrfs rescue super-recover /dev/sda" :
> > > > All supers are valid, no need to recover
> > > > Then "btrfs rescue zero-log /dev/sda", which produced a weird stacktrace...
> > > > Unable to find block group for 0
> > > > extent-tree.c:289: find_search_start: Assertion '1' failed.
> > > > btrfs[0x43e418]
> > > > btrfs(btrfs_reserve_extent+0x5c9)[0x4425df]
> > > > btrfs(btrfs_alloc_free_block+0x63[0x44297c]
> > > > btrfs(__btrfs_cow_block+0xfc[0x436636]
> > > > btrfs(btrfs_cow_block+0x8b)[0x436bd8]
> > > > btrfs[0x43ad82]
> > > > btrfs(btrfs_commit_transaction+0xb8)[0x43c5dc]
> > > > btrfs[0x42c0d4]btrfs(main+0x12f)[0x40a341]/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f1462d712e1]
> > > > btrfs(_start+0x2a)[0x40a37a]
> > > > Clearing log on /dev/sda, previous log_root 0, level 0
> > > > Finally I tried "btrfs rescue chunk-recover /dev/sda", which run on all 3 drives at the same time during 8+ hours...
> > > > It asks to rebuild some metadata tree, which I accepted (I did not saved the full output sorry) and it ended with the same stacktrace as above.
> > > > The only command left is "btrfs check --repair" but I afraid it might do more bad than good.
> > > > I'm running Debian 9 (still, because of some dependencies). My kernel is already backported : 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux
> > > > btrfs version : v4.7.3
> > > > I originally posted on reddit : https://www.reddit.com/r/btrfs/comments/g99v4v/nas_raid1_not_mounting_failed_to_read_block_groups/
> > > > Let me know if you need more information.
> > > > Nouts
> >
> > --
> > Hugo Mills | To an Englishman, 100 miles is a long way; to an
> > hugo@... carfax.org.uk | American, 100 years is a long time.
> > http://carfax.org.uk/ |
> > PGP: E2AB1DE4 | Earle Hitchner

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Chris Murphy]

On Thu, Apr 30, 2020 at 5:57 AM Nouts <nouts@protonmail.com> wrote:

> > [ 4645.402880] BTRFS info (device sdb): disk space caching is enabled
> > [ 4645.405687] BTRFS info (device sdb): has skinny extents
> > [ 4645.451484] BTRFS error (device sdb): failed to read block groups: -117
> > [ 4645.472062] BTRFS error (device sdb): open_ctree failed
> > mount: wrong fs type, bad option, bad superblock on /dev/sdb,missing codepage or helper program, or other error
> > In some cases useful info is found in syslog - trydmesg | tail or so.


> >
> > I attached you the smartctl result from the day before and the last scrub report I got from a month ago. From my understanding, it was ok.
> > I use hardlink (on the same partition/pool) and I deleted some data just the day before. I suspect my daily scrub routine triggered something that night and next day /home was gone.
> >
> > I can't scrub anymore as it's not mounted. Mounting with usebackuproot or degraded or ro produce the same error.
> > I tried "btrfs check /dev/sda" :
> > checking extents
> > leaf parent key incorrect 5909107507200
> > bad block 5909107507200
> > Errors found in extent allocation tree or chunk allocation
> > Checking filesystem on /dev/sda
> > UUID: 3720251f-ef92-4e21-bad0-eae1c97cff03

What do you get for:

btrfs insp dump-t -b 5909107507200 /dev/sda

> > Then "btrfs rescue zero-log /dev/sda", which produced a weird stacktrace...

btrfs-progs is really old


> > Finally I tried "btrfs rescue chunk-recover /dev/sda", which run on all 3 drives at the same time during 8+ hours...
> > It asks to rebuild some metadata tree, which I accepted (I did not saved the full output sorry) and it ended with the same stacktrace as above.
> >
> > The only command left is "btrfs check --repair" but I afraid it might do more bad than good.

With that version of btrfs-progs it's not advised.

> >
> > I'm running Debian 9 (still, because of some dependencies). My kernel is already backported : 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux
> > btrfs version : v4.7.3

I suggest finding newer btrfs-progs, 5.4 or better, or compiling it from git.
https://github.com/kdave//btrfs-progs

And then run:

btrfs check /dev/sda

Let's see what that says.


--
Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Thanks for your help. I compiled btrfs-progs v5.6 from github.

Here is the dump from /dev/sda : https://pastebin.com/e3YZxxsZ

And btrfs check returned an error instantly :
Opening filesystem to check...
ERROR: child eb corrupted: parent bytenr=5923702292480 item=2 parent level=2 child level=0
ERROR: failed to read block groups: Input/output error
ERROR: cannot open file system


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, April 30, 2020 7:38 PM, Chris Murphy <lists@colorremedies.com> wrote:

> On Thu, Apr 30, 2020 at 5:57 AM Nouts nouts@protonmail.com wrote:
>
> > > [ 4645.402880] BTRFS info (device sdb): disk space caching is enabled
> > > [ 4645.405687] BTRFS info (device sdb): has skinny extents
> > > [ 4645.451484] BTRFS error (device sdb): failed to read block groups: -117
> > > [ 4645.472062] BTRFS error (device sdb): open_ctree failed
> > > mount: wrong fs type, bad option, bad superblock on /dev/sdb,missing codepage or helper program, or other error
> > > In some cases useful info is found in syslog - trydmesg | tail or so.
>
> > > I attached you the smartctl result from the day before and the last scrub report I got from a month ago. From my understanding, it was ok.
> > > I use hardlink (on the same partition/pool) and I deleted some data just the day before. I suspect my daily scrub routine triggered something that night and next day /home was gone.
> > > I can't scrub anymore as it's not mounted. Mounting with usebackuproot or degraded or ro produce the same error.
> > > I tried "btrfs check /dev/sda" :
> > > checking extents
> > > leaf parent key incorrect 5909107507200
> > > bad block 5909107507200
> > > Errors found in extent allocation tree or chunk allocation
> > > Checking filesystem on /dev/sda
> > > UUID: 3720251f-ef92-4e21-bad0-eae1c97cff03
>
> What do you get for:
>
> btrfs insp dump-t -b 5909107507200 /dev/sda
>
> > > Then "btrfs rescue zero-log /dev/sda", which produced a weird stacktrace...
>
> btrfs-progs is really old
>
> > > Finally I tried "btrfs rescue chunk-recover /dev/sda", which run on all 3 drives at the same time during 8+ hours...
> > > It asks to rebuild some metadata tree, which I accepted (I did not saved the full output sorry) and it ended with the same stacktrace as above.
> > > The only command left is "btrfs check --repair" but I afraid it might do more bad than good.
>
> With that version of btrfs-progs it's not advised.
>
> > > I'm running Debian 9 (still, because of some dependencies). My kernel is already backported : 4.19.0-0.bpo.6-amd64 #1 SMP Debian 4.19.67-2+deb10u2~bpo9+1 (2019-11-12) x86_64 GNU/Linux
> > > btrfs version : v4.7.3
>
> I suggest finding newer btrfs-progs, 5.4 or better, or compiling it from git.
> https://github.com/kdave//btrfs-progs
>
> And then run:
>
> btrfs check /dev/sda
>
> Let's see what that says.
>
>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Chris Murphy]

On Thu, Apr 30, 2020 at 12:40 PM Nouts <nouts@protonmail.com> wrote:
>
> Thanks for your help. I compiled btrfs-progs v5.6 from github.
>
> Here is the dump from /dev/sda : https://pastebin.com/e3YZxxsZ
>
> And btrfs check returned an error instantly :
> Opening filesystem to check...
> ERROR: child eb corrupted: parent bytenr=5923702292480 item=2 parent level=2 child level=0
> ERROR: failed to read block groups: Input/output error
> ERROR: cannot open file system

Ok try:
btrfs insp dump-t -b 5923702292480 --follow /dev/sda

Qu might have an idea. But in the meantime, my suggestion is to try to
mount with a newer kernel. In order try:

normal mount
mount -o ro
mount -o ro,degraded
mount -o ro,degraded,usebackuproot

And if any works, at least you can update backups in case this file
system can't be fixed.


-- 
Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Here is the dump --follow : https://pastebin.com/yx13mDfB
Sadly, none of "mount" worked.

I'm on debian 9, already using backported Kernel.
For Kernel 5.4 or 5.6, I would have to use debian testing repo...
I'll see what I can do without breaking the rest of my system :)


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, April 30, 2020 8:46 PM, Chris Murphy <lists@colorremedies.com> wrote:

> On Thu, Apr 30, 2020 at 12:40 PM Nouts nouts@protonmail.com wrote:
>
> > Thanks for your help. I compiled btrfs-progs v5.6 from github.
> > Here is the dump from /dev/sda : https://pastebin.com/e3YZxxsZ
> > And btrfs check returned an error instantly :
> > Opening filesystem to check...
> > ERROR: child eb corrupted: parent bytenr=5923702292480 item=2 parent level=2 child level=0
> > ERROR: failed to read block groups: Input/output error
> > ERROR: cannot open file system
>
> Ok try:
> btrfs insp dump-t -b 5923702292480 --follow /dev/sda
>
> Qu might have an idea. But in the meantime, my suggestion is to try to
> mount with a newer kernel. In order try:
>
> normal mount
> mount -o ro
> mount -o ro,degraded
> mount -o ro,degraded,usebackuproot
>
> And if any works, at least you can update backups in case this file
> system can't be fixed.
>
>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Chris Murphy]

On Thu, Apr 30, 2020 at 1:05 PM Nouts <nouts@protonmail.com> wrote:
>
> Here is the dump --follow : https://pastebin.com/yx13mDfB
> Sadly, none of "mount" worked.

Yep. I can't tell from the check error or the dump output, whether
it's worth trying chunk recover again with the latest progs, or check
with --repair. But either of those makes changes to the file system,
so best to get advice from a developer first and in the meantime
refresh backups.

I don't know that a newer kernel will help in this case, because even
the latest progs complains. And the output from check doesn't give
enough information to know what to try next or if it's hopeless.

-- 
Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

For the record, I ran check and check --repair with the new btrfs-progs. Both returned this error :

Opening filesystem to check...
ERROR: child eb corrupted: parent bytenr=5923702292480 item=2 parent level=2 child level=0
ERROR: failed to read block groups: Input/output error
ERROR: cannot open file system

I'm gonna wipe the drive and restore a backup as I've lost hope recovering and can't wait any longer.
Thanks a lot for your help !

Nouts

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, April 30, 2020 9:18 PM, Chris Murphy <lists@colorremedies.com> wrote:

> On Thu, Apr 30, 2020 at 1:05 PM Nouts nouts@protonmail.com wrote:
>
> > Here is the dump --follow : https://pastebin.com/yx13mDfB
> > Sadly, none of "mount" worked.
>
> Yep. I can't tell from the check error or the dump output, whether
> it's worth trying chunk recover again with the latest progs, or check
> with --repair. But either of those makes changes to the file system,
> so best to get advice from a developer first and in the meantime
> refresh backups.
>
> I don't know that a newer kernel will help in this case, because even
> the latest progs complains. And the output from check doesn't give
> enough information to know what to try next or if it's hopeless.
>
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Well, I changed my mind. I'll wait a few more days, for the beginning of the next week, if you have any magical command left :)
I'm not sure to understand it but can a "restore" command dump all of the data in a new readable disc ?




‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, May 2, 2020 4:55 PM, Nouts <nouts@protonmail.com> wrote:

> For the record, I ran check and check --repair with the new btrfs-progs. Both returned this error :
>
> Opening filesystem to check...
> ERROR: child eb corrupted: parent bytenr=5923702292480 item=2 parent level=2 child level=0
> ERROR: failed to read block groups: Input/output error
> ERROR: cannot open file system
>
> I'm gonna wipe the drive and restore a backup as I've lost hope recovering and can't wait any longer.
> Thanks a lot for your help !
>
> Nouts
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, April 30, 2020 9:18 PM, Chris Murphy lists@colorremedies.com wrote:
>
> > On Thu, Apr 30, 2020 at 1:05 PM Nouts nouts@protonmail.com wrote:
> >
> > > Here is the dump --follow : https://pastebin.com/yx13mDfB
> > > Sadly, none of "mount" worked.
> >
> > Yep. I can't tell from the check error or the dump output, whether
> > it's worth trying chunk recover again with the latest progs, or check
> > with --repair. But either of those makes changes to the file system,
> > so best to get advice from a developer first and in the meantime
> > refresh backups.
> > I don't know that a newer kernel will help in this case, because even
> > the latest progs complains. And the output from check doesn't give
> > enough information to know what to try next or if it's hopeless.
> >
> > Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Chris Murphy]

On Sat, May 2, 2020 at 9:51 AM Nouts <nouts@protonmail.com> wrote:
>
> Well, I changed my mind. I'll wait a few more days, for the beginning of the next week, if you have any magical command left :)
> I'm not sure to understand it but can a "restore" command dump all of the data in a new readable disc ?

Yes, that's the only way it works. It's an offline scraping tool. It
is possible for recovered files to be corrupt, the point of the tool
is to improve the chance of recovery. So it doesn't have the same
safeguards that Btrfs has, where EIO happens upon corruption being
detected.

https://btrfs.wiki.kernel.org/index.php/Restore


-- 
Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Hello again,


Do you think someone will be able to tell us more about reports I provided ?
I ordered a new drive that can fit all the data from a "restore", so I'll wait until it's delivered.

Nouts

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, May 2, 2020 7:53 PM, Chris Murphy <lists@colorremedies.com> wrote:

> On Sat, May 2, 2020 at 9:51 AM Nouts nouts@protonmail.com wrote:
>
> > Well, I changed my mind. I'll wait a few more days, for the beginning of the next week, if you have any magical command left :)
> > I'm not sure to understand it but can a "restore" command dump all of the data in a new readable disc ?
>
> Yes, that's the only way it works. It's an offline scraping tool. It
> is possible for recovered files to be corrupt, the point of the tool
> is to improve the chance of recovery. So it doesn't have the same
> safeguards that Btrfs has, where EIO happens upon corruption being
> detected.
>
> https://btrfs.wiki.kernel.org/index.php/Restore
>
>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Chris Murphy

Re: Troubleshoot help needed - RAID1 not mounting : failed to read block groups

[Nouts]

Also, I just read about SMR disk and noticed my RAID1 is build only on SMR disk, no CMR.
I also use hardlink a lot due to Radarr/Sonarr. The day before it all gone wrong, I deleted some data (deleting the original files), leaving only the second files (the hardlinked one).

Could the mix of both be responsible of my issue ?


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, May 6, 2020 9:31 PM, Nouts <nouts@protonmail.com> wrote:

> Hello again,
>
> Do you think someone will be able to tell us more about reports I provided ?
> I ordered a new drive that can fit all the data from a "restore", so I'll wait until it's delivered.
>
> Nouts
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Saturday, May 2, 2020 7:53 PM, Chris Murphy lists@colorremedies.com wrote:
>
> > On Sat, May 2, 2020 at 9:51 AM Nouts nouts@protonmail.com wrote:
> >
> > > Well, I changed my mind. I'll wait a few more days, for the beginning of the next week, if you have any magical command left :)
> > > I'm not sure to understand it but can a "restore" command dump all of the data in a new readable disc ?
> >
> > Yes, that's the only way it works. It's an offline scraping tool. It
> > is possible for recovered files to be corrupt, the point of the tool
> > is to improve the chance of recovery. So it doesn't have the same
> > safeguards that Btrfs has, where EIO happens upon corruption being
> > detected.
> > https://btrfs.wiki.kernel.org/index.php/Restore
> >
> > Chris Murphy