btrfs check help

btrfs check help

[Vincent Olivier]

Hi,

Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10.

Regards,

Vincent

[root@3dcpc5 ~]# btrfs check /dev/sdk
Checking filesystem on /dev/sdk
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents
checking free space cache
checking fs roots
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong
found 19328809638262 bytes used err is 1
total csum bytes: 18849042724
total tree bytes: 27389886464
total fs tree bytes: 4449746944
total extent tree bytes: 3075457024
btree space waste bytes: 2880474254
file data blocks allocated: 19430708535296
referenced 20123773407232

Re: btrfs check help

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 1836 bytes --]

On 2015-11-24 12:06, Vincent Olivier wrote:
> Hi,
>
> Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10.
You get bonus points for being on a reasonably up-to-date kernel and 
userspace :)

This is actually a pretty tame check result for a filesystem that's been 
through kernel panic. I think everything listed here is safe for check 
to fix, but I would suggest waiting until the devs provide opinions 
before actually running with --repair.  I would also suggest comparing 
results between the different devices in the FS, if things are 
drastically different, you may have issues that check can't fix on it's own.
> [root@3dcpc5 ~]# btrfs check /dev/sdk
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> checking free space cache
> checking fs roots
These next two lines are errors, but I'm not 100% certain if it's safe 
to have check fix them:
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
This next one is also an error, and I am fairly certain that it's safe 
to have check fix as long as the number at the end is not too big.
> found 19328809638262 bytes used err is 1
The rest is just reference info
> total csum bytes: 18849042724
> total tree bytes: 27389886464
> total fs tree bytes: 4449746944
> total extent tree bytes: 3075457024
> btree space waste bytes: 2880474254
The only other thing I know that's worth mentioning is that if the 
numbers on these next two lines don't match, you may be missing some 
writes from right before the crash.
> file data blocks allocated: 19430708535296
> referenced 20123773407232




[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3019 bytes --]

Re: btrfs check help

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote:
> On 2015-11-24 12:06, Vincent Olivier wrote:
> >Hi,
> >
> >Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10.
> You get bonus points for being on a reasonably up-to-date kernel and
> userspace :)
> 
> This is actually a pretty tame check result for a filesystem that's
> been through kernel panic. I think everything listed here is safe
> for check to fix, but I would suggest waiting until the devs provide
> opinions before actually running with --repair.  I would also
> suggest comparing results between the different devices in the FS,
> if things are drastically different, you may have issues that check
> can't fix on it's own.
> >[root@3dcpc5 ~]# btrfs check /dev/sdk
> >Checking filesystem on /dev/sdk
> >UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> >checking extents
> >checking free space cache
> >checking fs roots
> These next two lines are errors, but I'm not 100% certain if it's
> safe to have check fix them:
> >root 5 inode 1341670 errors 400, nbytes wrong
> >root 11406 inode 1341670 errors 400, nbytes wrong

   I think so yes.

> This next one is also an error, and I am fairly certain that it's
> safe to have check fix as long as the number at the end is not too
> big.
> >found 19328809638262 bytes used err is 1

   Agreed.

   Hugo.

> The rest is just reference info
> >total csum bytes: 18849042724
> >total tree bytes: 27389886464
> >total fs tree bytes: 4449746944
> >total extent tree bytes: 3075457024
> >btree space waste bytes: 2880474254
> The only other thing I know that's worth mentioning is that if the
> numbers on these next two lines don't match, you may be missing some
> writes from right before the crash.
> >file data blocks allocated: 19430708535296
> >referenced 20123773407232

-- 
Hugo Mills             | Great films about cricket: Umpire of the Rising Sun
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: btrfs check help

[Vincent Olivier]

I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected.

You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”.

Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks.

Vincent


Checking filesystem on /dev/sdk
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdp
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [O]
checking free space cache [o]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdi
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [.]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdq
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [o]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdh
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdm
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [O]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdj
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [.]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdo
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [O]
checking free space cache [.]
checking fs roots [o]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdg
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [o]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdn
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [O]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdl
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [o]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdc
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [O]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdr
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [O]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdf
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [o]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sde
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [.]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdd
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680
Checking filesystem on /dev/sdb
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents [o]
checking free space cache [.]
root 5 inode 1341670 errors 400, nbytes wrong
root 11406 inode 1341670 errors 400, nbytes wrong

found 19328980191604 bytes used err is 1
total csum bytes: 18849205856
total tree bytes: 27393392640
total fs tree bytes: 4452958208
total extent tree bytes: 3075571712
btree space waste bytes: 2881050910
file data blocks allocated: 19445786390528
referenced 20138885959680



> On Nov 24, 2015, at 3:32 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> 
> On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote:
>> On 2015-11-24 12:06, Vincent Olivier wrote:
>>> Hi,
>>> 
>>> Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10.
>> You get bonus points for being on a reasonably up-to-date kernel and
>> userspace :)
>> 
>> This is actually a pretty tame check result for a filesystem that's
>> been through kernel panic. I think everything listed here is safe
>> for check to fix, but I would suggest waiting until the devs provide
>> opinions before actually running with --repair.  I would also
>> suggest comparing results between the different devices in the FS,
>> if things are drastically different, you may have issues that check
>> can't fix on it's own.
>>> [root@3dcpc5 ~]# btrfs check /dev/sdk
>>> Checking filesystem on /dev/sdk
>>> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
>>> checking extents
>>> checking free space cache
>>> checking fs roots
>> These next two lines are errors, but I'm not 100% certain if it's
>> safe to have check fix them:
>>> root 5 inode 1341670 errors 400, nbytes wrong
>>> root 11406 inode 1341670 errors 400, nbytes wrong
> 
>   I think so yes.
> 
>> This next one is also an error, and I am fairly certain that it's
>> safe to have check fix as long as the number at the end is not too
>> big.
>>> found 19328809638262 bytes used err is 1
> 
>   Agreed.
> 
>   Hugo.
> 
>> The rest is just reference info
>>> total csum bytes: 18849042724
>>> total tree bytes: 27389886464
>>> total fs tree bytes: 4449746944
>>> total extent tree bytes: 3075457024
>>> btree space waste bytes: 2880474254
>> The only other thing I know that's worth mentioning is that if the
>> numbers on these next two lines don't match, you may be missing some
>> writes from right before the crash.
>>> file data blocks allocated: 19430708535296
>>> referenced 20123773407232
> 
> -- 
> Hugo Mills             | Great films about cricket: Umpire of the Rising Sun
> hugo@... carfax.org.uk |
> http://carfax.org.uk/  |
> PGP: E2AB1DE4          |

Re: btrfs check help

[Henk Slager]

[...]
> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks.
>
> Vincent
>
>
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
[...]

I just remember that I have seen this kind of error before; luckily, I
found the btrfs check output (august 2015) on some backup of an old
snapshot; In my case it was on a raid5 fs from november 2013, 7 small
txt files (all several 100 bytes) and the 7 errors are repeated for
about 10 snapshots. I did   # find . -inum <inode number>    to find
the files. 2 of the 7 were still in the latest/actual subvol and I
just recreated them.

The errors from the older snapshots are still there as far as I
remember from the last btrfs check I did (with kernel 4.3.0 tools
4.3.x). The fs is converted to raid10 since 3 months. As I also got
other fake errors (as in this
https://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg48325.html
), I won't run a repair until I see proof that this 'errors 400,
nbytes wrong' is a risk for file-server stability.
I just see that on an archive clone fs with this 10 old snapshots
(created via send|receive), there is no error.

In your case, it is likely just 1 small file in rootvol (5) and the
same allocation in other subvol (11406), so maybe you can fix this
like I did and don't run a '--repair'

Re: btrfs check help

[Qu Wenruo]

Vincent Olivier wrote on 2015/11/25 11:51 -0500:
> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected.
>
> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”.

There is really no need to do such thing, as btrfs is able to manage 
multiple device, calling btrfsck on any of them is enough as long as 
it's not hugely damaged.

>
> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks.

YES.

Inode nbytes fix is *VERY* safe as long as it's the only error.

Although it's not that convincing since the inode nbytes fix code is 
written by myself and authors always tend to believe their codes are 
good....
But at least, some other users with more complicated problem(with inode 
nbytes error) fixed it.

The last decision is still on you anyway.

Thanks,
Qu


>
> Vincent
>
>
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdp
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [O]
> checking free space cache [o]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdi
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [.]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdq
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [o]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdh
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdm
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [O]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdj
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [.]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdo
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [O]
> checking free space cache [.]
> checking fs roots [o]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdg
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [o]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdn
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [O]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdl
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [o]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdc
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [O]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdr
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [O]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdf
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [o]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sde
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [.]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdd
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
> Checking filesystem on /dev/sdb
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents [o]
> checking free space cache [.]
> root 5 inode 1341670 errors 400, nbytes wrong
> root 11406 inode 1341670 errors 400, nbytes wrong
>
> found 19328980191604 bytes used err is 1
> total csum bytes: 18849205856
> total tree bytes: 27393392640
> total fs tree bytes: 4452958208
> total extent tree bytes: 3075571712
> btree space waste bytes: 2881050910
> file data blocks allocated: 19445786390528
> referenced 20138885959680
>
>
>
>> On Nov 24, 2015, at 3:32 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
>>
>> On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote:
>>> On 2015-11-24 12:06, Vincent Olivier wrote:
>>>> Hi,
>>>>
>>>> Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10.
>>> You get bonus points for being on a reasonably up-to-date kernel and
>>> userspace :)
>>>
>>> This is actually a pretty tame check result for a filesystem that's
>>> been through kernel panic. I think everything listed here is safe
>>> for check to fix, but I would suggest waiting until the devs provide
>>> opinions before actually running with --repair.  I would also
>>> suggest comparing results between the different devices in the FS,
>>> if things are drastically different, you may have issues that check
>>> can't fix on it's own.
>>>> [root@3dcpc5 ~]# btrfs check /dev/sdk
>>>> Checking filesystem on /dev/sdk
>>>> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
>>>> checking extents
>>>> checking free space cache
>>>> checking fs roots
>>> These next two lines are errors, but I'm not 100% certain if it's
>>> safe to have check fix them:
>>>> root 5 inode 1341670 errors 400, nbytes wrong
>>>> root 11406 inode 1341670 errors 400, nbytes wrong
>>
>>    I think so yes.
>>
>>> This next one is also an error, and I am fairly certain that it's
>>> safe to have check fix as long as the number at the end is not too
>>> big.
>>>> found 19328809638262 bytes used err is 1
>>
>>    Agreed.
>>
>>    Hugo.
>>
>>> The rest is just reference info
>>>> total csum bytes: 18849042724
>>>> total tree bytes: 27389886464
>>>> total fs tree bytes: 4449746944
>>>> total extent tree bytes: 3075457024
>>>> btree space waste bytes: 2880474254
>>> The only other thing I know that's worth mentioning is that if the
>>> numbers on these next two lines don't match, you may be missing some
>>> writes from right before the crash.
>>>> file data blocks allocated: 19430708535296
>>>> referenced 20123773407232
>>
>> --
>> Hugo Mills             | Great films about cricket: Umpire of the Rising Sun
>> hugo@... carfax.org.uk |
>> http://carfax.org.uk/  |
>> PGP: E2AB1DE4          |
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>

Re: btrfs check help

[Vincent Olivier]

> On Nov 25, 2015, at 8:44 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
> 
> 
> 
> Vincent Olivier wrote on 2015/11/25 11:51 -0500:
>> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected.
>> 
>> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”.
> 
> There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged.
> 
>> 
>> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks.
> 
> YES.
> 
> Inode nbytes fix is *VERY* safe as long as it's the only error.
> 
> Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good....
> But at least, some other users with more complicated problem(with inode nbytes error) fixed it.
> 
> The last decision is still on you anyway.

I will do it on the first device from the “fi show” output and report.

Thanks,

Vincent

Re: btrfs check help

[Vincent Olivier]

> On Nov 26, 2015, at 10:03 PM, Vincent Olivier <vincent@up4.com> wrote:
> 
>> 
>> On Nov 25, 2015, at 8:44 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
>> 
>> 
>> 
>> Vincent Olivier wrote on 2015/11/25 11:51 -0500:
>>> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected.
>>> 
>>> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”.
>> 
>> There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged.
>> 
>>> 
>>> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks.
>> 
>> YES.
>> 
>> Inode nbytes fix is *VERY* safe as long as it's the only error.
>> 
>> Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good....
>> But at least, some other users with more complicated problem(with inode nbytes error) fixed it.
>> 
>> The last decision is still on you anyway.
> 
> I will do it on the first device from the “fi show” output and report.


ok this doesn’t look good. i ran —repair and check again and it looks even worse. please help.


[root@3dcpc5 ~]# btrfs check --repair /dev/sdk
enabling repair mode
Checking filesystem on /dev/sdk
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents
Fixed 0 roots.
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
reset nbytes for ino 1341670 root 5
reset nbytes for ino 1341670 root 11406
warning line 3653
checking csums
checking root refs
found 19343374874998 bytes used err is 0
total csum bytes: 18863243900
total tree bytes: 27413118976
total fs tree bytes: 4455694336
total extent tree bytes: 3077373952
btree space waste bytes: 2882193883
file data blocks allocated: 19461564538880
 referenced 20155355832320





root@3dcpc5 ~]# btrfs check /dev/sdk
Checking filesystem on /dev/sdk
UUID: 6a742786-070d-4557-9e67-c73b84967bf5
checking extents
checking free space cache
block group 53328591454208 has wrong amount of free space
failed to load free space cache for block group 53328591454208
block group 53329665196032 has wrong amount of free space
failed to load free space cache for block group 53329665196032
Wanted offset 58836887044096, found 58836887011328
Wanted offset 58836887044096, found 58836887011328
cache appears valid but isnt 58836887011328
Wanted offset 60505481887744, found 60505481805824
Wanted offset 60505481887744, found 60505481805824
cache appears valid but isnt 60505481805824
Wanted bytes 16384, found 81920 for off 60979001966592
Wanted bytes 1073725440, found 81920 for off 60979001966592
cache appears valid but isnt 60979001950208
Wanted offset 61297908056064, found 61297908006912
Wanted offset 61297908056064, found 61297908006912
cache appears valid but isnt 61297903271936
Wanted bytes 32768, found 16384 for off 61711301296128
Wanted bytes 1066319872, found 16384 for off 61711301296128
cache appears valid but isnt 61711293874176
There is no free space entry for 62691824041984-62691824058368
There is no free space entry for 62691824041984-62692693901312
cache appears valid but isnt 62691620159488
There is no free space entry for 63723505205248-63723505221632
There is no free space entry for 63723505205248-63724559794176
cache appears valid but isnt 63723486052352
Wanted bytes 32768, found 16384 for off 64746920902656
Wanted bytes 914849792, found 16384 for off 64746920902656
cache appears valid but isnt 64746762010624
There is no free space entry for 65770368401408-65770368434176
There is no free space entry for 65770368401408-65771111710720
cache appears valid but isnt 65770037968896
Wanted offset 66758954270720, found 66758954221568
Wanted offset 66758954270720, found 66758954221568
cache appears valid but isnt 66758954188800
block group 70204591702016 has wrong amount of free space
failed to load free space cache for block group 70204591702016
block group 70205665443840 has wrong amount of free space
failed to load free space cache for block group 70205665443840
block group 70206739185664 has wrong amount of free space
failed to load free space cache for block group 70206739185664
Wanted offset 70216543715328, found 70216543698944
Wanted offset 70216543715328, found 70216543698944
cache appears valid but isnt 70216537079808
Wanted offset 71025067474944, found 71025067409408
Wanted offset 71025067474944, found 71025067409408
cache appears valid but isnt 71025064673280
Wanted offset 71455641354240, found 71455641337856
Wanted offset 71455641354240, found 71455641337856
cache appears valid but isnt 71455635144704
block group 71662867316736 has wrong amount of free space
failed to load free space cache for block group 71662867316736
block group 71663941058560 has wrong amount of free space
failed to load free space cache for block group 71663941058560
There is no free space entry for 72725872967680-72725872984064
There is no free space entry for 72725872967680-72726945464320
cache appears valid but isnt 72725871722496
block group 73207981801472 has wrong amount of free space
failed to load free space cache for block group 73207981801472
found 19343374940534 bytes used err is -22
total csum bytes: 18863243900
total tree bytes: 27413184512
total fs tree bytes: 4455727104
total extent tree bytes: 3077406720
btree space waste bytes: 2882234096
file data blocks allocated: 19461573357568
 referenced 20155367563264
[root@3dcpc5 ~]# 

Re: btrfs check help

[Henk Slager]

My experience/interpretation of the 2 checks is that it is OK, see
some more comments inserted below. Hopefully a developer of
btrfs-progs can comment in more detail.

> [root@3dcpc5 ~]# btrfs check --repair /dev/sdk
> enabling repair mode
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> Fixed 0 roots.
> checking free space cache
> cache and super generation don't match, space cache will be invalidated
This might be there because of the crash earlier, but a cache
invalidation should not be a problem.
> checking fs roots
> reset nbytes for ino 1341670 root 5
> reset nbytes for ino 1341670 root 11406
At least the nbytes error seems to be fixed.
> warning line 3653
> checking csums
> checking root refs
> found 19343374874998 bytes used err is 0
> total csum bytes: 18863243900
> total tree bytes: 27413118976
> total fs tree bytes: 4455694336
> total extent tree bytes: 3077373952
> btree space waste bytes: 2882193883
> file data blocks allocated: 19461564538880
>  referenced 20155355832320

The second readonly check partly can't deal with the just invalidated
space cache I think (I assume you haven't mounted and/or/ used
read-write the filesystem in between), but even if the space cache
wouldn't be touched in the --repair check, my experience is that those
errors, like in dmesg on my system:
[38018.645187] BTRFS info (device sdi): The free space cache file
(6258971115520) is invalid. skip it
   will disappear over time when the filesystem is filled/used.
This particular error is from a backup fs where one disk had gone bad.
A btrfs replace still worked and just after that, I saw many of those
errors, but now after a few weeks they are mostly gone. I did not
explicitly unmount or check--repair the fs, I just had to reboot the
system for another reason.
Your kernel+tools is new enough, you probably want to have a look at
the 'Space cache control' options on the wiki:
https://btrfs.wiki.kernel.org/index.php/Mount_options
  before you decide what to do.

> root@3dcpc5 ~]# btrfs check /dev/sdk
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> checking free space cache
> block group 53328591454208 has wrong amount of free space
> failed to load free space cache for block group 53328591454208
> block group 53329665196032 has wrong amount of free space
> failed to load free space cache for block group 53329665196032
> Wanted offset 58836887044096, found 58836887011328
> Wanted offset 58836887044096, found 58836887011328
> cache appears valid but isnt 58836887011328
> Wanted offset 60505481887744, found 60505481805824
> Wanted offset 60505481887744, found 60505481805824
> cache appears valid but isnt 60505481805824
> Wanted bytes 16384, found 81920 for off 60979001966592
> Wanted bytes 1073725440, found 81920 for off 60979001966592
> cache appears valid but isnt 60979001950208
> Wanted offset 61297908056064, found 61297908006912
> Wanted offset 61297908056064, found 61297908006912
> cache appears valid but isnt 61297903271936
> Wanted bytes 32768, found 16384 for off 61711301296128
> Wanted bytes 1066319872, found 16384 for off 61711301296128
> cache appears valid but isnt 61711293874176
> There is no free space entry for 62691824041984-62691824058368
> There is no free space entry for 62691824041984-62692693901312
> cache appears valid but isnt 62691620159488
> There is no free space entry for 63723505205248-63723505221632
> There is no free space entry for 63723505205248-63724559794176
> cache appears valid but isnt 63723486052352
> Wanted bytes 32768, found 16384 for off 64746920902656
> Wanted bytes 914849792, found 16384 for off 64746920902656
> cache appears valid but isnt 64746762010624
> There is no free space entry for 65770368401408-65770368434176
> There is no free space entry for 65770368401408-65771111710720
> cache appears valid but isnt 65770037968896
> Wanted offset 66758954270720, found 66758954221568
> Wanted offset 66758954270720, found 66758954221568
> cache appears valid but isnt 66758954188800
> block group 70204591702016 has wrong amount of free space
> failed to load free space cache for block group 70204591702016
> block group 70205665443840 has wrong amount of free space
> failed to load free space cache for block group 70205665443840
> block group 70206739185664 has wrong amount of free space
> failed to load free space cache for block group 70206739185664
> Wanted offset 70216543715328, found 70216543698944
> Wanted offset 70216543715328, found 70216543698944
> cache appears valid but isnt 70216537079808
> Wanted offset 71025067474944, found 71025067409408
> Wanted offset 71025067474944, found 71025067409408
> cache appears valid but isnt 71025064673280
> Wanted offset 71455641354240, found 71455641337856
> Wanted offset 71455641354240, found 71455641337856
> cache appears valid but isnt 71455635144704
> block group 71662867316736 has wrong amount of free space
> failed to load free space cache for block group 71662867316736
> block group 71663941058560 has wrong amount of free space
> failed to load free space cache for block group 71663941058560
> There is no free space entry for 72725872967680-72725872984064
> There is no free space entry for 72725872967680-72726945464320
> cache appears valid but isnt 72725871722496
> block group 73207981801472 has wrong amount of free space
> failed to load free space cache for block group 73207981801472
> found 19343374940534 bytes used err is -22
> total csum bytes: 18863243900
> total tree bytes: 27413184512
> total fs tree bytes: 4455727104
> total extent tree bytes: 3077406720
> btree space waste bytes: 2882234096
> file data blocks allocated: 19461573357568
>  referenced 20155367563264

Re: btrfs check help

[Chris Murphy]

On Fri, Nov 27, 2015 at 4:25 AM, Vincent Olivier <vincent@up4.com> wrote:
>
> [root@3dcpc5 ~]# btrfs check --repair /dev/sdk
> enabling repair mode
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> Fixed 0 roots.
> checking free space cache
> cache and super generation don't match, space cache will be invalidated
> checking fs roots
> reset nbytes for ino 1341670 root 5
> reset nbytes for ino 1341670 root 11406
> warning line 3653


I'm not sure what that last line means.



> root@3dcpc5 ~]# btrfs check /dev/sdk
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> checking free space cache
> block group 53328591454208 has wrong amount of free space
> failed to load free space cache for block group 53328591454208
> block group 53329665196032 has wrong amount of free space
> failed to load free space cache for block group 53329665196032
> Wanted offset 58836887044096, found 58836887011328
> Wanted offset 58836887044096, found 58836887011328
> cache appears valid but isnt 58836887011328
> Wanted offset 60505481887744, found 60505481805824
> Wanted offset 60505481887744, found 60505481805824
> cache appears valid but isnt 60505481805824
> Wanted bytes 16384, found 81920 for off 60979001966592
> Wanted bytes 1073725440, found 81920 for off 60979001966592
> cache appears valid but isnt 60979001950208
> Wanted offset 61297908056064, found 61297908006912
> Wanted offset 61297908056064, found 61297908006912
> cache appears valid but isnt 61297903271936
> Wanted bytes 32768, found 16384 for off 61711301296128
> Wanted bytes 1066319872, found 16384 for off 61711301296128
> cache appears valid but isnt 61711293874176
> There is no free space entry for 62691824041984-62691824058368
> There is no free space entry for 62691824041984-62692693901312
> cache appears valid but isnt 62691620159488
> There is no free space entry for 63723505205248-63723505221632
> There is no free space entry for 63723505205248-63724559794176
> cache appears valid but isnt 63723486052352
> Wanted bytes 32768, found 16384 for off 64746920902656
> Wanted bytes 914849792, found 16384 for off 64746920902656
> cache appears valid but isnt 64746762010624
> There is no free space entry for 65770368401408-65770368434176
> There is no free space entry for 65770368401408-65771111710720
> cache appears valid but isnt 65770037968896
> Wanted offset 66758954270720, found 66758954221568
> Wanted offset 66758954270720, found 66758954221568
> cache appears valid but isnt 66758954188800
> block group 70204591702016 has wrong amount of free space
> failed to load free space cache for block group 70204591702016
> block group 70205665443840 has wrong amount of free space
> failed to load free space cache for block group 70205665443840
> block group 70206739185664 has wrong amount of free space
> failed to load free space cache for block group 70206739185664
> Wanted offset 70216543715328, found 70216543698944
> Wanted offset 70216543715328, found 70216543698944
> cache appears valid but isnt 70216537079808
> Wanted offset 71025067474944, found 71025067409408
> Wanted offset 71025067474944, found 71025067409408
> cache appears valid but isnt 71025064673280
> Wanted offset 71455641354240, found 71455641337856
> Wanted offset 71455641354240, found 71455641337856
> cache appears valid but isnt 71455635144704
> block group 71662867316736 has wrong amount of free space
> failed to load free space cache for block group 71662867316736
> block group 71663941058560 has wrong amount of free space
> failed to load free space cache for block group 71663941058560
> There is no free space entry for 72725872967680-72725872984064
> There is no free space entry for 72725872967680-72726945464320
> cache appears valid but isnt 72725871722496
> block group 73207981801472 has wrong amount of free space
> failed to load free space cache for block group 73207981801472
> found 19343374940534 bytes used err is -22
> total csum bytes: 18863243900
> total tree bytes: 27413184512
> total fs tree bytes: 4455727104
> total extent tree bytes: 3077406720
> btree space waste bytes: 2882234096
> file data blocks allocated: 19461573357568
>  referenced 20155367563264


Except for the bytes used err is -22, I think this is just
acknowledging that the space caches are invalid, i.e. not a surprise.
It should get rebuilt at mount time, depending on the size of the file
system, it might take a while (?).


-- 
Chris Murphy

Re: btrfs check help

[Qu Wenruo]

Vincent Olivier wrote on 2015/11/27 06:25 -0500:
>
>> On Nov 26, 2015, at 10:03 PM, Vincent Olivier <vincent@up4.com> wrote:
>>
>>>
>>> On Nov 25, 2015, at 8:44 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote:
>>>
>>>
>>>
>>> Vincent Olivier wrote on 2015/11/25 11:51 -0500:
>>>> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected.
>>>>
>>>> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”.
>>>
>>> There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged.
>>>
>>>>
>>>> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks.
>>>
>>> YES.
>>>
>>> Inode nbytes fix is *VERY* safe as long as it's the only error.
>>>
>>> Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good....
>>> But at least, some other users with more complicated problem(with inode nbytes error) fixed it.
>>>
>>> The last decision is still on you anyway.
>>
>> I will do it on the first device from the “fi show” output and report.
>
>
> ok this doesn’t look good. i ran —repair and check again and it looks even worse. please help.
>
>
> [root@3dcpc5 ~]# btrfs check --repair /dev/sdk
> enabling repair mode
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> Fixed 0 roots.
> checking free space cache
> cache and super generation don't match, space cache will be invalidated
> checking fs roots
> reset nbytes for ino 1341670 root 5
> reset nbytes for ino 1341670 root 11406

As mentioned by other guys, inode nbytes seems to be fixed.

But to make it sure, if the inode is a directory or a normal file?

> warning line 3653

Seems to be a unexpected warning.
The subvolume root seems to be shared by other subvolume.

It may be one corner case for inode nbytes repair code.
But it seems no harm yet.

> checking csums
> checking root refs
> found 19343374874998 bytes used err is 0
> total csum bytes: 18863243900
> total tree bytes: 27413118976
> total fs tree bytes: 4455694336
> total extent tree bytes: 3077373952
> btree space waste bytes: 2882193883
> file data blocks allocated: 19461564538880
>   referenced 20155355832320
>
>
>
>
>
> root@3dcpc5 ~]# btrfs check /dev/sdk
> Checking filesystem on /dev/sdk
> UUID: 6a742786-070d-4557-9e67-c73b84967bf5
> checking extents
> checking free space cache
> block group 53328591454208 has wrong amount of free space
> failed to load free space cache for block group 53328591454208
> block group 53329665196032 has wrong amount of free space
> failed to load free space cache for block group 53329665196032
> Wanted offset 58836887044096, found 58836887011328
> Wanted offset 58836887044096, found 58836887011328
> cache appears valid but isnt 58836887011328
> Wanted offset 60505481887744, found 60505481805824
> Wanted offset 60505481887744, found 60505481805824
> cache appears valid but isnt 60505481805824
> Wanted bytes 16384, found 81920 for off 60979001966592
> Wanted bytes 1073725440, found 81920 for off 60979001966592
> cache appears valid but isnt 60979001950208
> Wanted offset 61297908056064, found 61297908006912
> Wanted offset 61297908056064, found 61297908006912
> cache appears valid but isnt 61297903271936
> Wanted bytes 32768, found 16384 for off 61711301296128
> Wanted bytes 1066319872, found 16384 for off 61711301296128
> cache appears valid but isnt 61711293874176
> There is no free space entry for 62691824041984-62691824058368
> There is no free space entry for 62691824041984-62692693901312
> cache appears valid but isnt 62691620159488
> There is no free space entry for 63723505205248-63723505221632
> There is no free space entry for 63723505205248-63724559794176
> cache appears valid but isnt 63723486052352
> Wanted bytes 32768, found 16384 for off 64746920902656
> Wanted bytes 914849792, found 16384 for off 64746920902656
> cache appears valid but isnt 64746762010624
> There is no free space entry for 65770368401408-65770368434176
> There is no free space entry for 65770368401408-65771111710720
> cache appears valid but isnt 65770037968896
> Wanted offset 66758954270720, found 66758954221568
> Wanted offset 66758954270720, found 66758954221568
> cache appears valid but isnt 66758954188800
> block group 70204591702016 has wrong amount of free space
> failed to load free space cache for block group 70204591702016
> block group 70205665443840 has wrong amount of free space
> failed to load free space cache for block group 70205665443840
> block group 70206739185664 has wrong amount of free space
> failed to load free space cache for block group 70206739185664
> Wanted offset 70216543715328, found 70216543698944
> Wanted offset 70216543715328, found 70216543698944
> cache appears valid but isnt 70216537079808
> Wanted offset 71025067474944, found 71025067409408
> Wanted offset 71025067474944, found 71025067409408
> cache appears valid but isnt 71025064673280
> Wanted offset 71455641354240, found 71455641337856
> Wanted offset 71455641354240, found 71455641337856
> cache appears valid but isnt 71455635144704
> block group 71662867316736 has wrong amount of free space
> failed to load free space cache for block group 71662867316736
> block group 71663941058560 has wrong amount of free space
> failed to load free space cache for block group 71663941058560
> There is no free space entry for 72725872967680-72725872984064
> There is no free space entry for 72725872967680-72726945464320
> cache appears valid but isnt 72725871722496
> block group 73207981801472 has wrong amount of free space
> failed to load free space cache for block group 73207981801472
> found 19343374940534 bytes used err is -22

Just free space cache mismatch.
Quite common for btrfsck, as it doesn't modify free space cache and 
alloced/freed some space.

It seems quite scary but in fact won't be a big problem.
The behavior can be improved for btrfsck, if it found old free space 
cache, just clean it to ensure good output.

To fix the problem, mount the filesystem with "-o clear_cache" to 
cleanup the wrong space cache.

Thanks,
Qu

> total csum bytes: 18863243900
> total tree bytes: 27413184512
> total fs tree bytes: 4455727104
> total extent tree bytes: 3077406720
> btree space waste bytes: 2882234096
> file data blocks allocated: 19461573357568
>   referenced 20155367563264
> [root@3dcpc5 ~]# --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>