* btrfs check help @ 2015-11-24 17:06 Vincent Olivier 2015-11-24 20:28 ` Austin S Hemmelgarn 0 siblings, 1 reply; 11+ messages in thread From: Vincent Olivier @ 2015-11-24 17:06 UTC (permalink / raw) To: linux-btrfs Hi, Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. Regards, Vincent [root@3dcpc5 ~]# btrfs check /dev/sdk Checking filesystem on /dev/sdk UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents checking free space cache checking fs roots root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328809638262 bytes used err is 1 total csum bytes: 18849042724 total tree bytes: 27389886464 total fs tree bytes: 4449746944 total extent tree bytes: 3075457024 btree space waste bytes: 2880474254 file data blocks allocated: 19430708535296 referenced 20123773407232 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-24 17:06 btrfs check help Vincent Olivier @ 2015-11-24 20:28 ` Austin S Hemmelgarn 2015-11-24 20:32 ` Hugo Mills 0 siblings, 1 reply; 11+ messages in thread From: Austin S Hemmelgarn @ 2015-11-24 20:28 UTC (permalink / raw) To: Vincent Olivier, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 1836 bytes --] On 2015-11-24 12:06, Vincent Olivier wrote: > Hi, > > Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. You get bonus points for being on a reasonably up-to-date kernel and userspace :) This is actually a pretty tame check result for a filesystem that's been through kernel panic. I think everything listed here is safe for check to fix, but I would suggest waiting until the devs provide opinions before actually running with --repair. I would also suggest comparing results between the different devices in the FS, if things are drastically different, you may have issues that check can't fix on it's own. > [root@3dcpc5 ~]# btrfs check /dev/sdk > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > checking free space cache > checking fs roots These next two lines are errors, but I'm not 100% certain if it's safe to have check fix them: > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong This next one is also an error, and I am fairly certain that it's safe to have check fix as long as the number at the end is not too big. > found 19328809638262 bytes used err is 1 The rest is just reference info > total csum bytes: 18849042724 > total tree bytes: 27389886464 > total fs tree bytes: 4449746944 > total extent tree bytes: 3075457024 > btree space waste bytes: 2880474254 The only other thing I know that's worth mentioning is that if the numbers on these next two lines don't match, you may be missing some writes from right before the crash. > file data blocks allocated: 19430708535296 > referenced 20123773407232 [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 3019 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-24 20:28 ` Austin S Hemmelgarn @ 2015-11-24 20:32 ` Hugo Mills 2015-11-25 16:51 ` Vincent Olivier 0 siblings, 1 reply; 11+ messages in thread From: Hugo Mills @ 2015-11-24 20:32 UTC (permalink / raw) To: Austin S Hemmelgarn; +Cc: Vincent Olivier, linux-btrfs [-- Attachment #1: Type: text/plain, Size: 2113 bytes --] On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote: > On 2015-11-24 12:06, Vincent Olivier wrote: > >Hi, > > > >Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. > You get bonus points for being on a reasonably up-to-date kernel and > userspace :) > > This is actually a pretty tame check result for a filesystem that's > been through kernel panic. I think everything listed here is safe > for check to fix, but I would suggest waiting until the devs provide > opinions before actually running with --repair. I would also > suggest comparing results between the different devices in the FS, > if things are drastically different, you may have issues that check > can't fix on it's own. > >[root@3dcpc5 ~]# btrfs check /dev/sdk > >Checking filesystem on /dev/sdk > >UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > >checking extents > >checking free space cache > >checking fs roots > These next two lines are errors, but I'm not 100% certain if it's > safe to have check fix them: > >root 5 inode 1341670 errors 400, nbytes wrong > >root 11406 inode 1341670 errors 400, nbytes wrong I think so yes. > This next one is also an error, and I am fairly certain that it's > safe to have check fix as long as the number at the end is not too > big. > >found 19328809638262 bytes used err is 1 Agreed. Hugo. > The rest is just reference info > >total csum bytes: 18849042724 > >total tree bytes: 27389886464 > >total fs tree bytes: 4449746944 > >total extent tree bytes: 3075457024 > >btree space waste bytes: 2880474254 > The only other thing I know that's worth mentioning is that if the > numbers on these next two lines don't match, you may be missing some > writes from right before the crash. > >file data blocks allocated: 19430708535296 > >referenced 20123773407232 -- Hugo Mills | Great films about cricket: Umpire of the Rising Sun hugo@... carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-24 20:32 ` Hugo Mills @ 2015-11-25 16:51 ` Vincent Olivier 2015-11-25 18:52 ` Henk Slager 2015-11-26 1:44 ` Qu Wenruo 0 siblings, 2 replies; 11+ messages in thread From: Vincent Olivier @ 2015-11-25 16:51 UTC (permalink / raw) To: linux-btrfs I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected. You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”. Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks. Vincent Checking filesystem on /dev/sdk UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdp UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [O] checking free space cache [o] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdi UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [.] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdq UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [o] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdh UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdm UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [O] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdj UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [.] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdo UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [O] checking free space cache [.] checking fs roots [o] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdg UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [o] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdn UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [O] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdl UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [o] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdc UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [O] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdr UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [O] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdf UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [o] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sde UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [.] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdd UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 Checking filesystem on /dev/sdb UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents [o] checking free space cache [.] root 5 inode 1341670 errors 400, nbytes wrong root 11406 inode 1341670 errors 400, nbytes wrong found 19328980191604 bytes used err is 1 total csum bytes: 18849205856 total tree bytes: 27393392640 total fs tree bytes: 4452958208 total extent tree bytes: 3075571712 btree space waste bytes: 2881050910 file data blocks allocated: 19445786390528 referenced 20138885959680 > On Nov 24, 2015, at 3:32 PM, Hugo Mills <hugo@carfax.org.uk> wrote: > > On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote: >> On 2015-11-24 12:06, Vincent Olivier wrote: >>> Hi, >>> >>> Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. >> You get bonus points for being on a reasonably up-to-date kernel and >> userspace :) >> >> This is actually a pretty tame check result for a filesystem that's >> been through kernel panic. I think everything listed here is safe >> for check to fix, but I would suggest waiting until the devs provide >> opinions before actually running with --repair. I would also >> suggest comparing results between the different devices in the FS, >> if things are drastically different, you may have issues that check >> can't fix on it's own. >>> [root@3dcpc5 ~]# btrfs check /dev/sdk >>> Checking filesystem on /dev/sdk >>> UUID: 6a742786-070d-4557-9e67-c73b84967bf5 >>> checking extents >>> checking free space cache >>> checking fs roots >> These next two lines are errors, but I'm not 100% certain if it's >> safe to have check fix them: >>> root 5 inode 1341670 errors 400, nbytes wrong >>> root 11406 inode 1341670 errors 400, nbytes wrong > > I think so yes. > >> This next one is also an error, and I am fairly certain that it's >> safe to have check fix as long as the number at the end is not too >> big. >>> found 19328809638262 bytes used err is 1 > > Agreed. > > Hugo. > >> The rest is just reference info >>> total csum bytes: 18849042724 >>> total tree bytes: 27389886464 >>> total fs tree bytes: 4449746944 >>> total extent tree bytes: 3075457024 >>> btree space waste bytes: 2880474254 >> The only other thing I know that's worth mentioning is that if the >> numbers on these next two lines don't match, you may be missing some >> writes from right before the crash. >>> file data blocks allocated: 19430708535296 >>> referenced 20123773407232 > > -- > Hugo Mills | Great films about cricket: Umpire of the Rising Sun > hugo@... carfax.org.uk | > http://carfax.org.uk/ | > PGP: E2AB1DE4 | ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-25 16:51 ` Vincent Olivier @ 2015-11-25 18:52 ` Henk Slager 2015-11-26 1:44 ` Qu Wenruo 1 sibling, 0 replies; 11+ messages in thread From: Henk Slager @ 2015-11-25 18:52 UTC (permalink / raw) To: Vincent Olivier; +Cc: linux-btrfs [...] > Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks. > > Vincent > > > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong [...] I just remember that I have seen this kind of error before; luckily, I found the btrfs check output (august 2015) on some backup of an old snapshot; In my case it was on a raid5 fs from november 2013, 7 small txt files (all several 100 bytes) and the 7 errors are repeated for about 10 snapshots. I did # find . -inum <inode number> to find the files. 2 of the 7 were still in the latest/actual subvol and I just recreated them. The errors from the older snapshots are still there as far as I remember from the last btrfs check I did (with kernel 4.3.0 tools 4.3.x). The fs is converted to raid10 since 3 months. As I also got other fake errors (as in this https://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg48325.html ), I won't run a repair until I see proof that this 'errors 400, nbytes wrong' is a risk for file-server stability. I just see that on an archive clone fs with this 10 old snapshots (created via send|receive), there is no error. In your case, it is likely just 1 small file in rootvol (5) and the same allocation in other subvol (11406), so maybe you can fix this like I did and don't run a '--repair' ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-25 16:51 ` Vincent Olivier 2015-11-25 18:52 ` Henk Slager @ 2015-11-26 1:44 ` Qu Wenruo 2015-11-27 3:03 ` Vincent Olivier 1 sibling, 1 reply; 11+ messages in thread From: Qu Wenruo @ 2015-11-26 1:44 UTC (permalink / raw) To: Vincent Olivier, linux-btrfs Vincent Olivier wrote on 2015/11/25 11:51 -0500: > I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected. > > You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”. There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged. > > Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks. YES. Inode nbytes fix is *VERY* safe as long as it's the only error. Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good.... But at least, some other users with more complicated problem(with inode nbytes error) fixed it. The last decision is still on you anyway. Thanks, Qu > > Vincent > > > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdp > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [O] > checking free space cache [o] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdi > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [.] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdq > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [o] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdh > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdm > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [O] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdj > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [.] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdo > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [O] > checking free space cache [.] > checking fs roots [o] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdg > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [o] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdn > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [O] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdl > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [o] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdc > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [O] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdr > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [O] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdf > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [o] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sde > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [.] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdd > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > Checking filesystem on /dev/sdb > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents [o] > checking free space cache [.] > root 5 inode 1341670 errors 400, nbytes wrong > root 11406 inode 1341670 errors 400, nbytes wrong > > found 19328980191604 bytes used err is 1 > total csum bytes: 18849205856 > total tree bytes: 27393392640 > total fs tree bytes: 4452958208 > total extent tree bytes: 3075571712 > btree space waste bytes: 2881050910 > file data blocks allocated: 19445786390528 > referenced 20138885959680 > > > >> On Nov 24, 2015, at 3:32 PM, Hugo Mills <hugo@carfax.org.uk> wrote: >> >> On Tue, Nov 24, 2015 at 03:28:28PM -0500, Austin S Hemmelgarn wrote: >>> On 2015-11-24 12:06, Vincent Olivier wrote: >>>> Hi, >>>> >>>> Woke up this morning with a kernel panic (for which I do not have details). Please find below the output for btrfs check. Is this normal ? What should I do ? Arch Linux 4.2.5. Btrfs-utils 4.3.1. 17x4TB RAID10. >>> You get bonus points for being on a reasonably up-to-date kernel and >>> userspace :) >>> >>> This is actually a pretty tame check result for a filesystem that's >>> been through kernel panic. I think everything listed here is safe >>> for check to fix, but I would suggest waiting until the devs provide >>> opinions before actually running with --repair. I would also >>> suggest comparing results between the different devices in the FS, >>> if things are drastically different, you may have issues that check >>> can't fix on it's own. >>>> [root@3dcpc5 ~]# btrfs check /dev/sdk >>>> Checking filesystem on /dev/sdk >>>> UUID: 6a742786-070d-4557-9e67-c73b84967bf5 >>>> checking extents >>>> checking free space cache >>>> checking fs roots >>> These next two lines are errors, but I'm not 100% certain if it's >>> safe to have check fix them: >>>> root 5 inode 1341670 errors 400, nbytes wrong >>>> root 11406 inode 1341670 errors 400, nbytes wrong >> >> I think so yes. >> >>> This next one is also an error, and I am fairly certain that it's >>> safe to have check fix as long as the number at the end is not too >>> big. >>>> found 19328809638262 bytes used err is 1 >> >> Agreed. >> >> Hugo. >> >>> The rest is just reference info >>>> total csum bytes: 18849042724 >>>> total tree bytes: 27389886464 >>>> total fs tree bytes: 4449746944 >>>> total extent tree bytes: 3075457024 >>>> btree space waste bytes: 2880474254 >>> The only other thing I know that's worth mentioning is that if the >>> numbers on these next two lines don't match, you may be missing some >>> writes from right before the crash. >>>> file data blocks allocated: 19430708535296 >>>> referenced 20123773407232 >> >> -- >> Hugo Mills | Great films about cricket: Umpire of the Rising Sun >> hugo@... carfax.org.uk | >> http://carfax.org.uk/ | >> PGP: E2AB1DE4 | > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-26 1:44 ` Qu Wenruo @ 2015-11-27 3:03 ` Vincent Olivier 2015-11-27 11:25 ` Vincent Olivier 0 siblings, 1 reply; 11+ messages in thread From: Vincent Olivier @ 2015-11-27 3:03 UTC (permalink / raw) To: linux-btrfs > On Nov 25, 2015, at 8:44 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote: > > > > Vincent Olivier wrote on 2015/11/25 11:51 -0500: >> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected. >> >> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”. > > There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged. > >> >> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks. > > YES. > > Inode nbytes fix is *VERY* safe as long as it's the only error. > > Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good.... > But at least, some other users with more complicated problem(with inode nbytes error) fixed it. > > The last decision is still on you anyway. I will do it on the first device from the “fi show” output and report. Thanks, Vincent ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-27 3:03 ` Vincent Olivier @ 2015-11-27 11:25 ` Vincent Olivier 2015-11-27 16:32 ` Henk Slager ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Vincent Olivier @ 2015-11-27 11:25 UTC (permalink / raw) To: linux-btrfs > On Nov 26, 2015, at 10:03 PM, Vincent Olivier <vincent@up4.com> wrote: > >> >> On Nov 25, 2015, at 8:44 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote: >> >> >> >> Vincent Olivier wrote on 2015/11/25 11:51 -0500: >>> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected. >>> >>> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”. >> >> There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged. >> >>> >>> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks. >> >> YES. >> >> Inode nbytes fix is *VERY* safe as long as it's the only error. >> >> Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good.... >> But at least, some other users with more complicated problem(with inode nbytes error) fixed it. >> >> The last decision is still on you anyway. > > I will do it on the first device from the “fi show” output and report. ok this doesn’t look good. i ran —repair and check again and it looks even worse. please help. [root@3dcpc5 ~]# btrfs check --repair /dev/sdk enabling repair mode Checking filesystem on /dev/sdk UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents Fixed 0 roots. checking free space cache cache and super generation don't match, space cache will be invalidated checking fs roots reset nbytes for ino 1341670 root 5 reset nbytes for ino 1341670 root 11406 warning line 3653 checking csums checking root refs found 19343374874998 bytes used err is 0 total csum bytes: 18863243900 total tree bytes: 27413118976 total fs tree bytes: 4455694336 total extent tree bytes: 3077373952 btree space waste bytes: 2882193883 file data blocks allocated: 19461564538880 referenced 20155355832320 root@3dcpc5 ~]# btrfs check /dev/sdk Checking filesystem on /dev/sdk UUID: 6a742786-070d-4557-9e67-c73b84967bf5 checking extents checking free space cache block group 53328591454208 has wrong amount of free space failed to load free space cache for block group 53328591454208 block group 53329665196032 has wrong amount of free space failed to load free space cache for block group 53329665196032 Wanted offset 58836887044096, found 58836887011328 Wanted offset 58836887044096, found 58836887011328 cache appears valid but isnt 58836887011328 Wanted offset 60505481887744, found 60505481805824 Wanted offset 60505481887744, found 60505481805824 cache appears valid but isnt 60505481805824 Wanted bytes 16384, found 81920 for off 60979001966592 Wanted bytes 1073725440, found 81920 for off 60979001966592 cache appears valid but isnt 60979001950208 Wanted offset 61297908056064, found 61297908006912 Wanted offset 61297908056064, found 61297908006912 cache appears valid but isnt 61297903271936 Wanted bytes 32768, found 16384 for off 61711301296128 Wanted bytes 1066319872, found 16384 for off 61711301296128 cache appears valid but isnt 61711293874176 There is no free space entry for 62691824041984-62691824058368 There is no free space entry for 62691824041984-62692693901312 cache appears valid but isnt 62691620159488 There is no free space entry for 63723505205248-63723505221632 There is no free space entry for 63723505205248-63724559794176 cache appears valid but isnt 63723486052352 Wanted bytes 32768, found 16384 for off 64746920902656 Wanted bytes 914849792, found 16384 for off 64746920902656 cache appears valid but isnt 64746762010624 There is no free space entry for 65770368401408-65770368434176 There is no free space entry for 65770368401408-65771111710720 cache appears valid but isnt 65770037968896 Wanted offset 66758954270720, found 66758954221568 Wanted offset 66758954270720, found 66758954221568 cache appears valid but isnt 66758954188800 block group 70204591702016 has wrong amount of free space failed to load free space cache for block group 70204591702016 block group 70205665443840 has wrong amount of free space failed to load free space cache for block group 70205665443840 block group 70206739185664 has wrong amount of free space failed to load free space cache for block group 70206739185664 Wanted offset 70216543715328, found 70216543698944 Wanted offset 70216543715328, found 70216543698944 cache appears valid but isnt 70216537079808 Wanted offset 71025067474944, found 71025067409408 Wanted offset 71025067474944, found 71025067409408 cache appears valid but isnt 71025064673280 Wanted offset 71455641354240, found 71455641337856 Wanted offset 71455641354240, found 71455641337856 cache appears valid but isnt 71455635144704 block group 71662867316736 has wrong amount of free space failed to load free space cache for block group 71662867316736 block group 71663941058560 has wrong amount of free space failed to load free space cache for block group 71663941058560 There is no free space entry for 72725872967680-72725872984064 There is no free space entry for 72725872967680-72726945464320 cache appears valid but isnt 72725871722496 block group 73207981801472 has wrong amount of free space failed to load free space cache for block group 73207981801472 found 19343374940534 bytes used err is -22 total csum bytes: 18863243900 total tree bytes: 27413184512 total fs tree bytes: 4455727104 total extent tree bytes: 3077406720 btree space waste bytes: 2882234096 file data blocks allocated: 19461573357568 referenced 20155367563264 [root@3dcpc5 ~]# ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-27 11:25 ` Vincent Olivier @ 2015-11-27 16:32 ` Henk Slager 2015-11-27 20:25 ` Chris Murphy 2015-11-30 1:54 ` Qu Wenruo 2 siblings, 0 replies; 11+ messages in thread From: Henk Slager @ 2015-11-27 16:32 UTC (permalink / raw) To: Vincent Olivier; +Cc: linux-btrfs My experience/interpretation of the 2 checks is that it is OK, see some more comments inserted below. Hopefully a developer of btrfs-progs can comment in more detail. > [root@3dcpc5 ~]# btrfs check --repair /dev/sdk > enabling repair mode > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > Fixed 0 roots. > checking free space cache > cache and super generation don't match, space cache will be invalidated This might be there because of the crash earlier, but a cache invalidation should not be a problem. > checking fs roots > reset nbytes for ino 1341670 root 5 > reset nbytes for ino 1341670 root 11406 At least the nbytes error seems to be fixed. > warning line 3653 > checking csums > checking root refs > found 19343374874998 bytes used err is 0 > total csum bytes: 18863243900 > total tree bytes: 27413118976 > total fs tree bytes: 4455694336 > total extent tree bytes: 3077373952 > btree space waste bytes: 2882193883 > file data blocks allocated: 19461564538880 > referenced 20155355832320 The second readonly check partly can't deal with the just invalidated space cache I think (I assume you haven't mounted and/or/ used read-write the filesystem in between), but even if the space cache wouldn't be touched in the --repair check, my experience is that those errors, like in dmesg on my system: [38018.645187] BTRFS info (device sdi): The free space cache file (6258971115520) is invalid. skip it will disappear over time when the filesystem is filled/used. This particular error is from a backup fs where one disk had gone bad. A btrfs replace still worked and just after that, I saw many of those errors, but now after a few weeks they are mostly gone. I did not explicitly unmount or check--repair the fs, I just had to reboot the system for another reason. Your kernel+tools is new enough, you probably want to have a look at the 'Space cache control' options on the wiki: https://btrfs.wiki.kernel.org/index.php/Mount_options before you decide what to do. > root@3dcpc5 ~]# btrfs check /dev/sdk > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > checking free space cache > block group 53328591454208 has wrong amount of free space > failed to load free space cache for block group 53328591454208 > block group 53329665196032 has wrong amount of free space > failed to load free space cache for block group 53329665196032 > Wanted offset 58836887044096, found 58836887011328 > Wanted offset 58836887044096, found 58836887011328 > cache appears valid but isnt 58836887011328 > Wanted offset 60505481887744, found 60505481805824 > Wanted offset 60505481887744, found 60505481805824 > cache appears valid but isnt 60505481805824 > Wanted bytes 16384, found 81920 for off 60979001966592 > Wanted bytes 1073725440, found 81920 for off 60979001966592 > cache appears valid but isnt 60979001950208 > Wanted offset 61297908056064, found 61297908006912 > Wanted offset 61297908056064, found 61297908006912 > cache appears valid but isnt 61297903271936 > Wanted bytes 32768, found 16384 for off 61711301296128 > Wanted bytes 1066319872, found 16384 for off 61711301296128 > cache appears valid but isnt 61711293874176 > There is no free space entry for 62691824041984-62691824058368 > There is no free space entry for 62691824041984-62692693901312 > cache appears valid but isnt 62691620159488 > There is no free space entry for 63723505205248-63723505221632 > There is no free space entry for 63723505205248-63724559794176 > cache appears valid but isnt 63723486052352 > Wanted bytes 32768, found 16384 for off 64746920902656 > Wanted bytes 914849792, found 16384 for off 64746920902656 > cache appears valid but isnt 64746762010624 > There is no free space entry for 65770368401408-65770368434176 > There is no free space entry for 65770368401408-65771111710720 > cache appears valid but isnt 65770037968896 > Wanted offset 66758954270720, found 66758954221568 > Wanted offset 66758954270720, found 66758954221568 > cache appears valid but isnt 66758954188800 > block group 70204591702016 has wrong amount of free space > failed to load free space cache for block group 70204591702016 > block group 70205665443840 has wrong amount of free space > failed to load free space cache for block group 70205665443840 > block group 70206739185664 has wrong amount of free space > failed to load free space cache for block group 70206739185664 > Wanted offset 70216543715328, found 70216543698944 > Wanted offset 70216543715328, found 70216543698944 > cache appears valid but isnt 70216537079808 > Wanted offset 71025067474944, found 71025067409408 > Wanted offset 71025067474944, found 71025067409408 > cache appears valid but isnt 71025064673280 > Wanted offset 71455641354240, found 71455641337856 > Wanted offset 71455641354240, found 71455641337856 > cache appears valid but isnt 71455635144704 > block group 71662867316736 has wrong amount of free space > failed to load free space cache for block group 71662867316736 > block group 71663941058560 has wrong amount of free space > failed to load free space cache for block group 71663941058560 > There is no free space entry for 72725872967680-72725872984064 > There is no free space entry for 72725872967680-72726945464320 > cache appears valid but isnt 72725871722496 > block group 73207981801472 has wrong amount of free space > failed to load free space cache for block group 73207981801472 > found 19343374940534 bytes used err is -22 > total csum bytes: 18863243900 > total tree bytes: 27413184512 > total fs tree bytes: 4455727104 > total extent tree bytes: 3077406720 > btree space waste bytes: 2882234096 > file data blocks allocated: 19461573357568 > referenced 20155367563264 ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-27 11:25 ` Vincent Olivier 2015-11-27 16:32 ` Henk Slager @ 2015-11-27 20:25 ` Chris Murphy 2015-11-30 1:54 ` Qu Wenruo 2 siblings, 0 replies; 11+ messages in thread From: Chris Murphy @ 2015-11-27 20:25 UTC (permalink / raw) To: Vincent Olivier; +Cc: linux-btrfs On Fri, Nov 27, 2015 at 4:25 AM, Vincent Olivier <vincent@up4.com> wrote: > > [root@3dcpc5 ~]# btrfs check --repair /dev/sdk > enabling repair mode > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > Fixed 0 roots. > checking free space cache > cache and super generation don't match, space cache will be invalidated > checking fs roots > reset nbytes for ino 1341670 root 5 > reset nbytes for ino 1341670 root 11406 > warning line 3653 I'm not sure what that last line means. > root@3dcpc5 ~]# btrfs check /dev/sdk > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > checking free space cache > block group 53328591454208 has wrong amount of free space > failed to load free space cache for block group 53328591454208 > block group 53329665196032 has wrong amount of free space > failed to load free space cache for block group 53329665196032 > Wanted offset 58836887044096, found 58836887011328 > Wanted offset 58836887044096, found 58836887011328 > cache appears valid but isnt 58836887011328 > Wanted offset 60505481887744, found 60505481805824 > Wanted offset 60505481887744, found 60505481805824 > cache appears valid but isnt 60505481805824 > Wanted bytes 16384, found 81920 for off 60979001966592 > Wanted bytes 1073725440, found 81920 for off 60979001966592 > cache appears valid but isnt 60979001950208 > Wanted offset 61297908056064, found 61297908006912 > Wanted offset 61297908056064, found 61297908006912 > cache appears valid but isnt 61297903271936 > Wanted bytes 32768, found 16384 for off 61711301296128 > Wanted bytes 1066319872, found 16384 for off 61711301296128 > cache appears valid but isnt 61711293874176 > There is no free space entry for 62691824041984-62691824058368 > There is no free space entry for 62691824041984-62692693901312 > cache appears valid but isnt 62691620159488 > There is no free space entry for 63723505205248-63723505221632 > There is no free space entry for 63723505205248-63724559794176 > cache appears valid but isnt 63723486052352 > Wanted bytes 32768, found 16384 for off 64746920902656 > Wanted bytes 914849792, found 16384 for off 64746920902656 > cache appears valid but isnt 64746762010624 > There is no free space entry for 65770368401408-65770368434176 > There is no free space entry for 65770368401408-65771111710720 > cache appears valid but isnt 65770037968896 > Wanted offset 66758954270720, found 66758954221568 > Wanted offset 66758954270720, found 66758954221568 > cache appears valid but isnt 66758954188800 > block group 70204591702016 has wrong amount of free space > failed to load free space cache for block group 70204591702016 > block group 70205665443840 has wrong amount of free space > failed to load free space cache for block group 70205665443840 > block group 70206739185664 has wrong amount of free space > failed to load free space cache for block group 70206739185664 > Wanted offset 70216543715328, found 70216543698944 > Wanted offset 70216543715328, found 70216543698944 > cache appears valid but isnt 70216537079808 > Wanted offset 71025067474944, found 71025067409408 > Wanted offset 71025067474944, found 71025067409408 > cache appears valid but isnt 71025064673280 > Wanted offset 71455641354240, found 71455641337856 > Wanted offset 71455641354240, found 71455641337856 > cache appears valid but isnt 71455635144704 > block group 71662867316736 has wrong amount of free space > failed to load free space cache for block group 71662867316736 > block group 71663941058560 has wrong amount of free space > failed to load free space cache for block group 71663941058560 > There is no free space entry for 72725872967680-72725872984064 > There is no free space entry for 72725872967680-72726945464320 > cache appears valid but isnt 72725871722496 > block group 73207981801472 has wrong amount of free space > failed to load free space cache for block group 73207981801472 > found 19343374940534 bytes used err is -22 > total csum bytes: 18863243900 > total tree bytes: 27413184512 > total fs tree bytes: 4455727104 > total extent tree bytes: 3077406720 > btree space waste bytes: 2882234096 > file data blocks allocated: 19461573357568 > referenced 20155367563264 Except for the bytes used err is -22, I think this is just acknowledging that the space caches are invalid, i.e. not a surprise. It should get rebuilt at mount time, depending on the size of the file system, it might take a while (?). -- Chris Murphy ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: btrfs check help 2015-11-27 11:25 ` Vincent Olivier 2015-11-27 16:32 ` Henk Slager 2015-11-27 20:25 ` Chris Murphy @ 2015-11-30 1:54 ` Qu Wenruo 2 siblings, 0 replies; 11+ messages in thread From: Qu Wenruo @ 2015-11-30 1:54 UTC (permalink / raw) To: Vincent Olivier, linux-btrfs Vincent Olivier wrote on 2015/11/27 06:25 -0500: > >> On Nov 26, 2015, at 10:03 PM, Vincent Olivier <vincent@up4.com> wrote: >> >>> >>> On Nov 25, 2015, at 8:44 PM, Qu Wenruo <quwenruo@cn.fujitsu.com> wrote: >>> >>> >>> >>> Vincent Olivier wrote on 2015/11/25 11:51 -0500: >>>> I should probably point out that there is 64GB of RAM on this machine and it’s a dual Xeon processor (LGA2011-3) system. Also, there is only Btrfs served via Samba and the kernel panic was caused Btrfs (as per what I remember from the log on the screen just before I rebooted) and happened in the middle of the night when zero (0) client was connected. >>>> >>>> You will find below the full “btrfs check” log for each device in the order it is listed by “btrfs fi show”. >>> >>> There is really no need to do such thing, as btrfs is able to manage multiple device, calling btrfsck on any of them is enough as long as it's not hugely damaged. >>> >>>> >>>> Ca I get a strong confirmation that I should run with the “—repair” option on each device? Thanks. >>> >>> YES. >>> >>> Inode nbytes fix is *VERY* safe as long as it's the only error. >>> >>> Although it's not that convincing since the inode nbytes fix code is written by myself and authors always tend to believe their codes are good.... >>> But at least, some other users with more complicated problem(with inode nbytes error) fixed it. >>> >>> The last decision is still on you anyway. >> >> I will do it on the first device from the “fi show” output and report. > > > ok this doesn’t look good. i ran —repair and check again and it looks even worse. please help. > > > [root@3dcpc5 ~]# btrfs check --repair /dev/sdk > enabling repair mode > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > Fixed 0 roots. > checking free space cache > cache and super generation don't match, space cache will be invalidated > checking fs roots > reset nbytes for ino 1341670 root 5 > reset nbytes for ino 1341670 root 11406 As mentioned by other guys, inode nbytes seems to be fixed. But to make it sure, if the inode is a directory or a normal file? > warning line 3653 Seems to be a unexpected warning. The subvolume root seems to be shared by other subvolume. It may be one corner case for inode nbytes repair code. But it seems no harm yet. > checking csums > checking root refs > found 19343374874998 bytes used err is 0 > total csum bytes: 18863243900 > total tree bytes: 27413118976 > total fs tree bytes: 4455694336 > total extent tree bytes: 3077373952 > btree space waste bytes: 2882193883 > file data blocks allocated: 19461564538880 > referenced 20155355832320 > > > > > > root@3dcpc5 ~]# btrfs check /dev/sdk > Checking filesystem on /dev/sdk > UUID: 6a742786-070d-4557-9e67-c73b84967bf5 > checking extents > checking free space cache > block group 53328591454208 has wrong amount of free space > failed to load free space cache for block group 53328591454208 > block group 53329665196032 has wrong amount of free space > failed to load free space cache for block group 53329665196032 > Wanted offset 58836887044096, found 58836887011328 > Wanted offset 58836887044096, found 58836887011328 > cache appears valid but isnt 58836887011328 > Wanted offset 60505481887744, found 60505481805824 > Wanted offset 60505481887744, found 60505481805824 > cache appears valid but isnt 60505481805824 > Wanted bytes 16384, found 81920 for off 60979001966592 > Wanted bytes 1073725440, found 81920 for off 60979001966592 > cache appears valid but isnt 60979001950208 > Wanted offset 61297908056064, found 61297908006912 > Wanted offset 61297908056064, found 61297908006912 > cache appears valid but isnt 61297903271936 > Wanted bytes 32768, found 16384 for off 61711301296128 > Wanted bytes 1066319872, found 16384 for off 61711301296128 > cache appears valid but isnt 61711293874176 > There is no free space entry for 62691824041984-62691824058368 > There is no free space entry for 62691824041984-62692693901312 > cache appears valid but isnt 62691620159488 > There is no free space entry for 63723505205248-63723505221632 > There is no free space entry for 63723505205248-63724559794176 > cache appears valid but isnt 63723486052352 > Wanted bytes 32768, found 16384 for off 64746920902656 > Wanted bytes 914849792, found 16384 for off 64746920902656 > cache appears valid but isnt 64746762010624 > There is no free space entry for 65770368401408-65770368434176 > There is no free space entry for 65770368401408-65771111710720 > cache appears valid but isnt 65770037968896 > Wanted offset 66758954270720, found 66758954221568 > Wanted offset 66758954270720, found 66758954221568 > cache appears valid but isnt 66758954188800 > block group 70204591702016 has wrong amount of free space > failed to load free space cache for block group 70204591702016 > block group 70205665443840 has wrong amount of free space > failed to load free space cache for block group 70205665443840 > block group 70206739185664 has wrong amount of free space > failed to load free space cache for block group 70206739185664 > Wanted offset 70216543715328, found 70216543698944 > Wanted offset 70216543715328, found 70216543698944 > cache appears valid but isnt 70216537079808 > Wanted offset 71025067474944, found 71025067409408 > Wanted offset 71025067474944, found 71025067409408 > cache appears valid but isnt 71025064673280 > Wanted offset 71455641354240, found 71455641337856 > Wanted offset 71455641354240, found 71455641337856 > cache appears valid but isnt 71455635144704 > block group 71662867316736 has wrong amount of free space > failed to load free space cache for block group 71662867316736 > block group 71663941058560 has wrong amount of free space > failed to load free space cache for block group 71663941058560 > There is no free space entry for 72725872967680-72725872984064 > There is no free space entry for 72725872967680-72726945464320 > cache appears valid but isnt 72725871722496 > block group 73207981801472 has wrong amount of free space > failed to load free space cache for block group 73207981801472 > found 19343374940534 bytes used err is -22 Just free space cache mismatch. Quite common for btrfsck, as it doesn't modify free space cache and alloced/freed some space. It seems quite scary but in fact won't be a big problem. The behavior can be improved for btrfsck, if it found old free space cache, just clean it to ensure good output. To fix the problem, mount the filesystem with "-o clear_cache" to cleanup the wrong space cache. Thanks, Qu > total csum bytes: 18863243900 > total tree bytes: 27413184512 > total fs tree bytes: 4455727104 > total extent tree bytes: 3077406720 > btree space waste bytes: 2882234096 > file data blocks allocated: 19461573357568 > referenced 20155367563264 > [root@3dcpc5 ~]# -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2015-11-30 1:54 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-11-24 17:06 btrfs check help Vincent Olivier 2015-11-24 20:28 ` Austin S Hemmelgarn 2015-11-24 20:32 ` Hugo Mills 2015-11-25 16:51 ` Vincent Olivier 2015-11-25 18:52 ` Henk Slager 2015-11-26 1:44 ` Qu Wenruo 2015-11-27 3:03 ` Vincent Olivier 2015-11-27 11:25 ` Vincent Olivier 2015-11-27 16:32 ` Henk Slager 2015-11-27 20:25 ` Chris Murphy 2015-11-30 1:54 ` Qu Wenruo
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.