* btrfs volume can't see files in folder
@ 2022-03-31 19:29 Rosen Penev
2022-03-31 23:39 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Rosen Penev @ 2022-03-31 19:29 UTC (permalink / raw)
To: linux-btrfs
A specific folder has files in it. Directly accessing the path works
but ls in the directory returns empty.
Any way to fix this issue? I believe it happened after a btrfs
replace(failed drive in RAID5) + btrfs balance.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-03-31 19:29 btrfs volume can't see files in folder Rosen Penev
@ 2022-03-31 23:39 ` Qu Wenruo
2022-04-01 0:24 ` Rosen Penev
0 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2022-03-31 23:39 UTC (permalink / raw)
To: Rosen Penev, linux-btrfs
On 2022/4/1 03:29, Rosen Penev wrote:
> A specific folder has files in it. Directly accessing the path works
> but ls in the directory returns empty.
>
> Any way to fix this issue? I believe it happened after a btrfs
> replace(failed drive in RAID5) + btrfs balance.
Btrfs check please.
It looks like an DIR_ITEM/DIR_INDEX corruption.
Thanks,
Qu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-03-31 23:39 ` Qu Wenruo
@ 2022-04-01 0:24 ` Rosen Penev
2022-04-01 0:59 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Rosen Penev @ 2022-04-01 0:24 UTC (permalink / raw)
To: Qu Wenruo; +Cc: linux-btrfs
On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2022/4/1 03:29, Rosen Penev wrote:
> > A specific folder has files in it. Directly accessing the path works
> > but ls in the directory returns empty.
> >
> > Any way to fix this issue? I believe it happened after a btrfs
> > replace(failed drive in RAID5) + btrfs balance.
>
> Btrfs check please.
btrfs check --force /dev/sda
Opening filesystem to check...
Checking filesystem on /dev/sda
UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 139616845824
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 146059296768
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 3183842689024
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 3184916430848
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 3185990172672
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 3187063914496
btrfs: space cache generation (1084391) does not match inode (1084389)
failed to load free space cache for block group 3190318694400
block group 4885757034496 has wrong amount of free space, free space
cache has 286720 block group has 290816
failed to load free space cache for block group 4885757034496
block group 4898641936384 has wrong amount of free space, free space
cache has 36864 block group has 53248
failed to load free space cache for block group 4898641936384
block group 4953402769408 has wrong amount of free space, free space
cache has 262144 block group has 274432
failed to load free space cache for block group 4953402769408
block group 5478462521344 has wrong amount of free space, free space
cache has 716800 block group has 729088
failed to load free space cache for block group 5478462521344
block group 5484904972288 has wrong amount of free space, free space
cache has 811008 block group has 819200
failed to load free space cache for block group 5484904972288
[4/7] checking fs roots
It's currently stuck on that last one.
>
> It looks like an DIR_ITEM/DIR_INDEX corruption.
>
> Thanks,
> Qu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 0:24 ` Rosen Penev
@ 2022-04-01 0:59 ` Qu Wenruo
2022-04-01 2:48 ` Rosen Penev
0 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2022-04-01 0:59 UTC (permalink / raw)
To: Rosen Penev, Qu Wenruo; +Cc: linux-btrfs
On 2022/4/1 08:24, Rosen Penev wrote:
> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>>
>> On 2022/4/1 03:29, Rosen Penev wrote:
>>> A specific folder has files in it. Directly accessing the path works
>>> but ls in the directory returns empty.
>>>
>>> Any way to fix this issue? I believe it happened after a btrfs
>>> replace(failed drive in RAID5) + btrfs balance.
>>
>> Btrfs check please.
> btrfs check --force /dev/sda
Force is not recommended unless it's your root fs and you don't really
want to run btrfs check on an liveCD.
As sometimes it can report false positive if the fs is not mounted
read-only.
> Opening filesystem to check...
> Checking filesystem on /dev/sda
> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> [1/7] checking root items
> [2/7] checking extents
> [3/7] checking free space cache
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 139616845824
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 146059296768
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 3183842689024
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 3184916430848
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 3185990172672
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 3187063914496
> btrfs: space cache generation (1084391) does not match inode (1084389)
> failed to load free space cache for block group 3190318694400
> block group 4885757034496 has wrong amount of free space, free space
> cache has 286720 block group has 290816
> failed to load free space cache for block group 4885757034496
> block group 4898641936384 has wrong amount of free space, free space
> cache has 36864 block group has 53248
> failed to load free space cache for block group 4898641936384
> block group 4953402769408 has wrong amount of free space, free space
> cache has 262144 block group has 274432
> failed to load free space cache for block group 4953402769408
> block group 5478462521344 has wrong amount of free space, free space
> cache has 716800 block group has 729088
> failed to load free space cache for block group 5478462521344
> block group 5484904972288 has wrong amount of free space, free space
> cache has 811008 block group has 819200
> failed to load free space cache for block group 5484904972288
> [4/7] checking fs roots
>
> It's currently stuck on that last one.
If the fs is pretty large, it can take quite some time.
Thanks,
Qu
>
>>
>> It looks like an DIR_ITEM/DIR_INDEX corruption.
>>
>> Thanks,
>> Qu
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 0:59 ` Qu Wenruo
@ 2022-04-01 2:48 ` Rosen Penev
2022-04-01 2:53 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Rosen Penev @ 2022-04-01 2:48 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs
On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
>
>
>
> On 2022/4/1 08:24, Rosen Penev wrote:
> > On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>
> >>
> >>
> >> On 2022/4/1 03:29, Rosen Penev wrote:
> >>> A specific folder has files in it. Directly accessing the path works
> >>> but ls in the directory returns empty.
> >>>
> >>> Any way to fix this issue? I believe it happened after a btrfs
> >>> replace(failed drive in RAID5) + btrfs balance.
> >>
> >> Btrfs check please.
> > btrfs check --force /dev/sda
>
> Force is not recommended unless it's your root fs and you don't really
> want to run btrfs check on an liveCD.
Same result without force and unmounted:
btrfs check /dev/sda
Opening filesystem to check...
Checking filesystem on /dev/sda
UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
block group 4885757034496 has wrong amount of free space, free space
cache has 286720 block group has 290816
failed to load free space cache for block group 4885757034496
block group 4898641936384 has wrong amount of free space, free space
cache has 36864 block group has 53248
failed to load free space cache for block group 4898641936384
block group 4953402769408 has wrong amount of free space, free space
cache has 262144 block group has 274432
failed to load free space cache for block group 4953402769408
block group 5478462521344 has wrong amount of free space, free space
cache has 716800 block group has 729088
failed to load free space cache for block group 5478462521344
block group 5484904972288 has wrong amount of free space, free space
cache has 811008 block group has 819200
failed to load free space cache for block group 5484904972288
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
found 2689565679616 bytes used, no error found
total csum bytes: 2620609300
total tree bytes: 5374935040
total fs tree bytes: 1737539584
total extent tree bytes: 511115264
btree space waste bytes: 889131100
file data blocks allocated: 41913072627712
referenced 2675025698816
>
> As sometimes it can report false positive if the fs is not mounted
> read-only.
>
> > Opening filesystem to check...
> > Checking filesystem on /dev/sda
> > UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> > [1/7] checking root items
> > [2/7] checking extents
> > [3/7] checking free space cache
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 139616845824
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 146059296768
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 3183842689024
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 3184916430848
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 3185990172672
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 3187063914496
> > btrfs: space cache generation (1084391) does not match inode (1084389)
> > failed to load free space cache for block group 3190318694400
> > block group 4885757034496 has wrong amount of free space, free space
> > cache has 286720 block group has 290816
> > failed to load free space cache for block group 4885757034496
> > block group 4898641936384 has wrong amount of free space, free space
> > cache has 36864 block group has 53248
> > failed to load free space cache for block group 4898641936384
> > block group 4953402769408 has wrong amount of free space, free space
> > cache has 262144 block group has 274432
> > failed to load free space cache for block group 4953402769408
> > block group 5478462521344 has wrong amount of free space, free space
> > cache has 716800 block group has 729088
> > failed to load free space cache for block group 5478462521344
> > block group 5484904972288 has wrong amount of free space, free space
> > cache has 811008 block group has 819200
> > failed to load free space cache for block group 5484904972288
> > [4/7] checking fs roots
> >
> > It's currently stuck on that last one.
>
> If the fs is pretty large, it can take quite some time.
>
> Thanks,
> Qu
>
> >
> >>
> >> It looks like an DIR_ITEM/DIR_INDEX corruption.
> >>
> >> Thanks,
> >> Qu
> >
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 2:48 ` Rosen Penev
@ 2022-04-01 2:53 ` Qu Wenruo
2022-04-01 3:05 ` Rosen Penev
0 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2022-04-01 2:53 UTC (permalink / raw)
To: Rosen Penev, Qu Wenruo; +Cc: linux-btrfs
On 2022/4/1 10:48, Rosen Penev wrote:
> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
>>
>>
>>
>> On 2022/4/1 08:24, Rosen Penev wrote:
>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2022/4/1 03:29, Rosen Penev wrote:
>>>>> A specific folder has files in it. Directly accessing the path works
>>>>> but ls in the directory returns empty.
>>>>>
>>>>> Any way to fix this issue? I believe it happened after a btrfs
>>>>> replace(failed drive in RAID5) + btrfs balance.
>>>>
>>>> Btrfs check please.
>>> btrfs check --force /dev/sda
>>
>> Force is not recommended unless it's your root fs and you don't really
>> want to run btrfs check on an liveCD.
> Same result without force and unmounted:
>
> btrfs check /dev/sda
> Opening filesystem to check...
> Checking filesystem on /dev/sda
> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> [1/7] checking root items
> [2/7] checking extents
> [3/7] checking free space cache
> block group 4885757034496 has wrong amount of free space, free space
> cache has 286720 block group has 290816
> failed to load free space cache for block group 4885757034496
> block group 4898641936384 has wrong amount of free space, free space
> cache has 36864 block group has 53248
> failed to load free space cache for block group 4898641936384
> block group 4953402769408 has wrong amount of free space, free space
> cache has 262144 block group has 274432
> failed to load free space cache for block group 4953402769408
> block group 5478462521344 has wrong amount of free space, free space
> cache has 716800 block group has 729088
> failed to load free space cache for block group 5478462521344
> block group 5484904972288 has wrong amount of free space, free space
> cache has 811008 block group has 819200
> failed to load free space cache for block group 5484904972288
Only non-critical free space cache problem, and kernel can detect and
rebuild them without problem.
> [4/7] checking fs roots
This is the most important part, and it turns no problem at all.
So at least your fs is completely fine.
It must be something else causing the problem.
Mind to provide the subvolume id and the inode number (`stat` command
can return the inode number) of the offending directory?
And some example command output when you can access the files inside the
directory but `ls -alh` shows nothing?
Thanks,
Qu
> [5/7] checking only csums items (without verifying data)
> [6/7] checking root refs
> [7/7] checking quota groups skipped (not enabled on this FS)
> found 2689565679616 bytes used, no error found
> total csum bytes: 2620609300
> total tree bytes: 5374935040
> total fs tree bytes: 1737539584
> total extent tree bytes: 511115264
> btree space waste bytes: 889131100
> file data blocks allocated: 41913072627712
> referenced 2675025698816
>
>>
>> As sometimes it can report false positive if the fs is not mounted
>> read-only.
>>
>>> Opening filesystem to check...
>>> Checking filesystem on /dev/sda
>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>> [1/7] checking root items
>>> [2/7] checking extents
>>> [3/7] checking free space cache
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 139616845824
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 146059296768
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 3183842689024
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 3184916430848
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 3185990172672
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 3187063914496
>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>> failed to load free space cache for block group 3190318694400
>>> block group 4885757034496 has wrong amount of free space, free space
>>> cache has 286720 block group has 290816
>>> failed to load free space cache for block group 4885757034496
>>> block group 4898641936384 has wrong amount of free space, free space
>>> cache has 36864 block group has 53248
>>> failed to load free space cache for block group 4898641936384
>>> block group 4953402769408 has wrong amount of free space, free space
>>> cache has 262144 block group has 274432
>>> failed to load free space cache for block group 4953402769408
>>> block group 5478462521344 has wrong amount of free space, free space
>>> cache has 716800 block group has 729088
>>> failed to load free space cache for block group 5478462521344
>>> block group 5484904972288 has wrong amount of free space, free space
>>> cache has 811008 block group has 819200
>>> failed to load free space cache for block group 5484904972288
>>> [4/7] checking fs roots
>>>
>>> It's currently stuck on that last one.
>>
>> If the fs is pretty large, it can take quite some time.
>>
>> Thanks,
>> Qu
>>
>>>
>>>>
>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
>>>>
>>>> Thanks,
>>>> Qu
>>>
>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 2:53 ` Qu Wenruo
@ 2022-04-01 3:05 ` Rosen Penev
2022-04-01 3:18 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Rosen Penev @ 2022-04-01 3:05 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs
On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2022/4/1 10:48, Rosen Penev wrote:
> > On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
> >>
> >>
> >>
> >> On 2022/4/1 08:24, Rosen Penev wrote:
> >>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 2022/4/1 03:29, Rosen Penev wrote:
> >>>>> A specific folder has files in it. Directly accessing the path works
> >>>>> but ls in the directory returns empty.
> >>>>>
> >>>>> Any way to fix this issue? I believe it happened after a btrfs
> >>>>> replace(failed drive in RAID5) + btrfs balance.
> >>>>
> >>>> Btrfs check please.
> >>> btrfs check --force /dev/sda
> >>
> >> Force is not recommended unless it's your root fs and you don't really
> >> want to run btrfs check on an liveCD.
> > Same result without force and unmounted:
> >
> > btrfs check /dev/sda
> > Opening filesystem to check...
> > Checking filesystem on /dev/sda
> > UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> > [1/7] checking root items
> > [2/7] checking extents
> > [3/7] checking free space cache
> > block group 4885757034496 has wrong amount of free space, free space
> > cache has 286720 block group has 290816
> > failed to load free space cache for block group 4885757034496
> > block group 4898641936384 has wrong amount of free space, free space
> > cache has 36864 block group has 53248
> > failed to load free space cache for block group 4898641936384
> > block group 4953402769408 has wrong amount of free space, free space
> > cache has 262144 block group has 274432
> > failed to load free space cache for block group 4953402769408
> > block group 5478462521344 has wrong amount of free space, free space
> > cache has 716800 block group has 729088
> > failed to load free space cache for block group 5478462521344
> > block group 5484904972288 has wrong amount of free space, free space
> > cache has 811008 block group has 819200
> > failed to load free space cache for block group 5484904972288
>
> Only non-critical free space cache problem, and kernel can detect and
> rebuild them without problem.
>
> > [4/7] checking fs roots
>
> This is the most important part, and it turns no problem at all.
>
> So at least your fs is completely fine.
>
>
> It must be something else causing the problem.
>
> Mind to provide the subvolume id and the inode number (`stat` command
> can return the inode number) of the offending directory?
stat .
File: .
Size: 0 Blocks: 0 IO Block: 4096 directory
Device: 44h/68d Inode: 259 Links: 1
Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
Access: 2022-03-31 19:58:36.577915854 -0700
Modify: 2022-03-13 22:57:55.825138581 -0700
Change: 2022-03-13 22:57:55.825138581 -0700
Birth: 2020-05-16 20:14:31.577476911 -0700
btrfs subvolume list shows:
ID 259 gen 1084405 top level 5 path Torrents
>
> And some example command output when you can access the files inside the
> directory but `ls -alh` shows nothing?
shows . and .. . That's it.
The reason I know there are files here is because my torrent client is
currently seeding them. If I change the Catagory (which moves the
files elsewhere), the files show up in the given directory.
>
> Thanks,
> Qu
>
> > [5/7] checking only csums items (without verifying data)
> > [6/7] checking root refs
> > [7/7] checking quota groups skipped (not enabled on this FS)
> > found 2689565679616 bytes used, no error found
> > total csum bytes: 2620609300
> > total tree bytes: 5374935040
> > total fs tree bytes: 1737539584
> > total extent tree bytes: 511115264
> > btree space waste bytes: 889131100
> > file data blocks allocated: 41913072627712
> > referenced 2675025698816
> >
> >>
> >> As sometimes it can report false positive if the fs is not mounted
> >> read-only.
> >>
> >>> Opening filesystem to check...
> >>> Checking filesystem on /dev/sda
> >>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>> [1/7] checking root items
> >>> [2/7] checking extents
> >>> [3/7] checking free space cache
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 139616845824
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 146059296768
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 3183842689024
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 3184916430848
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 3185990172672
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 3187063914496
> >>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>> failed to load free space cache for block group 3190318694400
> >>> block group 4885757034496 has wrong amount of free space, free space
> >>> cache has 286720 block group has 290816
> >>> failed to load free space cache for block group 4885757034496
> >>> block group 4898641936384 has wrong amount of free space, free space
> >>> cache has 36864 block group has 53248
> >>> failed to load free space cache for block group 4898641936384
> >>> block group 4953402769408 has wrong amount of free space, free space
> >>> cache has 262144 block group has 274432
> >>> failed to load free space cache for block group 4953402769408
> >>> block group 5478462521344 has wrong amount of free space, free space
> >>> cache has 716800 block group has 729088
> >>> failed to load free space cache for block group 5478462521344
> >>> block group 5484904972288 has wrong amount of free space, free space
> >>> cache has 811008 block group has 819200
> >>> failed to load free space cache for block group 5484904972288
> >>> [4/7] checking fs roots
> >>>
> >>> It's currently stuck on that last one.
> >>
> >> If the fs is pretty large, it can take quite some time.
> >>
> >> Thanks,
> >> Qu
> >>
> >>>
> >>>>
> >>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
> >>>>
> >>>> Thanks,
> >>>> Qu
> >>>
> >>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 3:05 ` Rosen Penev
@ 2022-04-01 3:18 ` Qu Wenruo
2022-04-01 3:25 ` Rosen Penev
0 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2022-04-01 3:18 UTC (permalink / raw)
To: Rosen Penev; +Cc: Qu Wenruo, linux-btrfs
On 2022/4/1 11:05, Rosen Penev wrote:
> On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>>
>> On 2022/4/1 10:48, Rosen Penev wrote:
>>> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2022/4/1 08:24, Rosen Penev wrote:
>>>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2022/4/1 03:29, Rosen Penev wrote:
>>>>>>> A specific folder has files in it. Directly accessing the path works
>>>>>>> but ls in the directory returns empty.
>>>>>>>
>>>>>>> Any way to fix this issue? I believe it happened after a btrfs
>>>>>>> replace(failed drive in RAID5) + btrfs balance.
>>>>>>
>>>>>> Btrfs check please.
>>>>> btrfs check --force /dev/sda
>>>>
>>>> Force is not recommended unless it's your root fs and you don't really
>>>> want to run btrfs check on an liveCD.
>>> Same result without force and unmounted:
>>>
>>> btrfs check /dev/sda
>>> Opening filesystem to check...
>>> Checking filesystem on /dev/sda
>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>> [1/7] checking root items
>>> [2/7] checking extents
>>> [3/7] checking free space cache
>>> block group 4885757034496 has wrong amount of free space, free space
>>> cache has 286720 block group has 290816
>>> failed to load free space cache for block group 4885757034496
>>> block group 4898641936384 has wrong amount of free space, free space
>>> cache has 36864 block group has 53248
>>> failed to load free space cache for block group 4898641936384
>>> block group 4953402769408 has wrong amount of free space, free space
>>> cache has 262144 block group has 274432
>>> failed to load free space cache for block group 4953402769408
>>> block group 5478462521344 has wrong amount of free space, free space
>>> cache has 716800 block group has 729088
>>> failed to load free space cache for block group 5478462521344
>>> block group 5484904972288 has wrong amount of free space, free space
>>> cache has 811008 block group has 819200
>>> failed to load free space cache for block group 5484904972288
>>
>> Only non-critical free space cache problem, and kernel can detect and
>> rebuild them without problem.
>>
>>> [4/7] checking fs roots
>>
>> This is the most important part, and it turns no problem at all.
>>
>> So at least your fs is completely fine.
>>
>>
>> It must be something else causing the problem.
>>
>> Mind to provide the subvolume id and the inode number (`stat` command
>> can return the inode number) of the offending directory?
> stat .
> File: .
> Size: 0 Blocks: 0 IO Block: 4096 directory
> Device: 44h/68d Inode: 259 Links: 1
> Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
> Access: 2022-03-31 19:58:36.577915854 -0700
> Modify: 2022-03-13 22:57:55.825138581 -0700
> Change: 2022-03-13 22:57:55.825138581 -0700
> Birth: 2020-05-16 20:14:31.577476911 -0700
>
> btrfs subvolume list shows:
> ID 259 gen 1084405 top level 5 path Torrents
>>
>> And some example command output when you can access the files inside the
>> directory but `ls -alh` shows nothing?
> shows . and .. . That's it.
Mind to dump the the following contents?
NOTE: this will include file names, feel free to censor filenames if needed:
# btrfs ins dump-tree -t 259 <mnt> | grep "(259 " -A8
This will dump all info related to inode 259 inside subvolume "Torrents".
If there is really nothing but the subvolume itself, it may be something
else.
Thanks,
Qu
>
> The reason I know there are files here is because my torrent client is
> currently seeding them. If I change the Catagory (which moves the
> files elsewhere), the files show up in the given directory.
>>
>> Thanks,
>> Qu
>>
>>> [5/7] checking only csums items (without verifying data)
>>> [6/7] checking root refs
>>> [7/7] checking quota groups skipped (not enabled on this FS)
>>> found 2689565679616 bytes used, no error found
>>> total csum bytes: 2620609300
>>> total tree bytes: 5374935040
>>> total fs tree bytes: 1737539584
>>> total extent tree bytes: 511115264
>>> btree space waste bytes: 889131100
>>> file data blocks allocated: 41913072627712
>>> referenced 2675025698816
>>>
>>>>
>>>> As sometimes it can report false positive if the fs is not mounted
>>>> read-only.
>>>>
>>>>> Opening filesystem to check...
>>>>> Checking filesystem on /dev/sda
>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>>>> [1/7] checking root items
>>>>> [2/7] checking extents
>>>>> [3/7] checking free space cache
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 139616845824
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 146059296768
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 3183842689024
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 3184916430848
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 3185990172672
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 3187063914496
>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>> failed to load free space cache for block group 3190318694400
>>>>> block group 4885757034496 has wrong amount of free space, free space
>>>>> cache has 286720 block group has 290816
>>>>> failed to load free space cache for block group 4885757034496
>>>>> block group 4898641936384 has wrong amount of free space, free space
>>>>> cache has 36864 block group has 53248
>>>>> failed to load free space cache for block group 4898641936384
>>>>> block group 4953402769408 has wrong amount of free space, free space
>>>>> cache has 262144 block group has 274432
>>>>> failed to load free space cache for block group 4953402769408
>>>>> block group 5478462521344 has wrong amount of free space, free space
>>>>> cache has 716800 block group has 729088
>>>>> failed to load free space cache for block group 5478462521344
>>>>> block group 5484904972288 has wrong amount of free space, free space
>>>>> cache has 811008 block group has 819200
>>>>> failed to load free space cache for block group 5484904972288
>>>>> [4/7] checking fs roots
>>>>>
>>>>> It's currently stuck on that last one.
>>>>
>>>> If the fs is pretty large, it can take quite some time.
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>>
>>>>>>
>>>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>
>>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 3:18 ` Qu Wenruo
@ 2022-04-01 3:25 ` Rosen Penev
2022-04-01 5:05 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Rosen Penev @ 2022-04-01 3:25 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs
On Thu, Mar 31, 2022 at 8:18 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2022/4/1 11:05, Rosen Penev wrote:
> > On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>
> >>
> >>
> >> On 2022/4/1 10:48, Rosen Penev wrote:
> >>> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 2022/4/1 08:24, Rosen Penev wrote:
> >>>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 2022/4/1 03:29, Rosen Penev wrote:
> >>>>>>> A specific folder has files in it. Directly accessing the path works
> >>>>>>> but ls in the directory returns empty.
> >>>>>>>
> >>>>>>> Any way to fix this issue? I believe it happened after a btrfs
> >>>>>>> replace(failed drive in RAID5) + btrfs balance.
> >>>>>>
> >>>>>> Btrfs check please.
> >>>>> btrfs check --force /dev/sda
> >>>>
> >>>> Force is not recommended unless it's your root fs and you don't really
> >>>> want to run btrfs check on an liveCD.
> >>> Same result without force and unmounted:
> >>>
> >>> btrfs check /dev/sda
> >>> Opening filesystem to check...
> >>> Checking filesystem on /dev/sda
> >>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>> [1/7] checking root items
> >>> [2/7] checking extents
> >>> [3/7] checking free space cache
> >>> block group 4885757034496 has wrong amount of free space, free space
> >>> cache has 286720 block group has 290816
> >>> failed to load free space cache for block group 4885757034496
> >>> block group 4898641936384 has wrong amount of free space, free space
> >>> cache has 36864 block group has 53248
> >>> failed to load free space cache for block group 4898641936384
> >>> block group 4953402769408 has wrong amount of free space, free space
> >>> cache has 262144 block group has 274432
> >>> failed to load free space cache for block group 4953402769408
> >>> block group 5478462521344 has wrong amount of free space, free space
> >>> cache has 716800 block group has 729088
> >>> failed to load free space cache for block group 5478462521344
> >>> block group 5484904972288 has wrong amount of free space, free space
> >>> cache has 811008 block group has 819200
> >>> failed to load free space cache for block group 5484904972288
> >>
> >> Only non-critical free space cache problem, and kernel can detect and
> >> rebuild them without problem.
> >>
> >>> [4/7] checking fs roots
> >>
> >> This is the most important part, and it turns no problem at all.
> >>
> >> So at least your fs is completely fine.
> >>
> >>
> >> It must be something else causing the problem.
> >>
> >> Mind to provide the subvolume id and the inode number (`stat` command
> >> can return the inode number) of the offending directory?
> > stat .
> > File: .
> > Size: 0 Blocks: 0 IO Block: 4096 directory
> > Device: 44h/68d Inode: 259 Links: 1
> > Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
> > Access: 2022-03-31 19:58:36.577915854 -0700
> > Modify: 2022-03-13 22:57:55.825138581 -0700
> > Change: 2022-03-13 22:57:55.825138581 -0700
> > Birth: 2020-05-16 20:14:31.577476911 -0700
> >
> > btrfs subvolume list shows:
> > ID 259 gen 1084405 top level 5 path Torrents
> >>
> >> And some example command output when you can access the files inside the
> >> directory but `ls -alh` shows nothing?
> > shows . and .. . That's it.
>
> Mind to dump the the following contents?
> NOTE: this will include file names, feel free to censor filenames if needed:
>
> # btrfs ins dump-tree -t 259 <mnt> | grep "(259 " -A8
>
> This will dump all info related to inode 259 inside subvolume "Torrents".
Now it errors :)
btrfs ins dump-tree -t 259 /dev/sda | grep "(259 " -A8
file tree key (259 ROOT_ITEM 0)
node 3184169484288 level 2 items 37 free space 456 generation 1084405 owner 259
node 3184169484288 flags 0x1(WRITTEN) backref revision 1
fs uuid bfa267c0-df2c-45a6-ad88-9d76b3844326
chunk uuid 69378187-7a8d-42cc-a28d-a935000d8a94
key (256 INODE_ITEM 0) block 3184169500672 gen 1084405
key (1707 INODE_ITEM 0) block 3190334537728 gen 1081752
key (3045 INODE_ITEM 0) block 3184776314880 gen 1082176
key (7354 INODE_ITEM 0) block 3184775987200 gen 1082176
--
location key (259 INODE_ITEM 0) type DIR
transid 398 data_len 0 name_len 6
name: B
item 9 key (256 DIR_ITEM 2983476959) itemoff 15743 itemsize 35
location key (257 INODE_ITEM 0) type DIR
transid 398 data_len 0 name_len 5
name: A
item 10 key (256 DIR_ITEM 3061133479) itemoff 15682 itemsize 61
location key (372987 INODE_ITEM 0) type FILE
--
location key (259 INODE_ITEM 0) type DIR
transid 398 data_len 0 name_len 6
name: B
item 19 key (256 DIR_INDEX 5) itemoff 15286 itemsize 36
location key (260 INODE_ITEM 0) type DIR
transid 398 data_len 0 name_len 6
name: C
item 20 key (256 DIR_INDEX 6) itemoff 15250 itemsize 36
location key (261 INODE_ITEM 0) type DIR
--
item 30 key (259 INODE_ITEM 0) itemoff 13737 itemsize 160
generation 398 transid 1084405 size 0 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 176 flags 0x0(none)
atime 1648781916.577915854 (2022-03-31 19:58:36)
ctime 1647237475.825138581 (2022-03-13 22:57:55)
mtime 1647237475.825138581 (2022-03-13 22:57:55)
otime 1589685271.577476911 (2020-05-16 20:14:31)
item 31 key (259 INODE_REF 256) itemoff 13721 itemsize 16
index 4 namelen 6 name: B
item 32 key (260 INODE_ITEM 0) itemoff 13561 itemsize 160
generation 398 transid 1074067 size 56 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 405 flags 0x0(none)
atime 1648367720.152037253 (2022-03-27 00:55:20)
ctime 1646816503.726286540 (2022-03-09 01:01:43)
mtime 1646816503.726286540 (2022-03-09 01:01:43)
parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
Ignoring transid failure
parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
Ignoring transid failure
parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
Ignoring transid failure
ERROR: child eb corrupted: parent bytenr=3184315416576 item=4 parent
level=1 child bytenr=3184317464576 child level=2
Note that name B is the problematic one. I can access A and C just
fine. Interestingly enough there are many more directories.
>
> If there is really nothing but the subvolume itself, it may be something
> else.
>
> Thanks,
> Qu
> >
> > The reason I know there are files here is because my torrent client is
> > currently seeding them. If I change the Catagory (which moves the
> > files elsewhere), the files show up in the given directory.
> >>
> >> Thanks,
> >> Qu
> >>
> >>> [5/7] checking only csums items (without verifying data)
> >>> [6/7] checking root refs
> >>> [7/7] checking quota groups skipped (not enabled on this FS)
> >>> found 2689565679616 bytes used, no error found
> >>> total csum bytes: 2620609300
> >>> total tree bytes: 5374935040
> >>> total fs tree bytes: 1737539584
> >>> total extent tree bytes: 511115264
> >>> btree space waste bytes: 889131100
> >>> file data blocks allocated: 41913072627712
> >>> referenced 2675025698816
> >>>
> >>>>
> >>>> As sometimes it can report false positive if the fs is not mounted
> >>>> read-only.
> >>>>
> >>>>> Opening filesystem to check...
> >>>>> Checking filesystem on /dev/sda
> >>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>>>> [1/7] checking root items
> >>>>> [2/7] checking extents
> >>>>> [3/7] checking free space cache
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 139616845824
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 146059296768
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 3183842689024
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 3184916430848
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 3185990172672
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 3187063914496
> >>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>> failed to load free space cache for block group 3190318694400
> >>>>> block group 4885757034496 has wrong amount of free space, free space
> >>>>> cache has 286720 block group has 290816
> >>>>> failed to load free space cache for block group 4885757034496
> >>>>> block group 4898641936384 has wrong amount of free space, free space
> >>>>> cache has 36864 block group has 53248
> >>>>> failed to load free space cache for block group 4898641936384
> >>>>> block group 4953402769408 has wrong amount of free space, free space
> >>>>> cache has 262144 block group has 274432
> >>>>> failed to load free space cache for block group 4953402769408
> >>>>> block group 5478462521344 has wrong amount of free space, free space
> >>>>> cache has 716800 block group has 729088
> >>>>> failed to load free space cache for block group 5478462521344
> >>>>> block group 5484904972288 has wrong amount of free space, free space
> >>>>> cache has 811008 block group has 819200
> >>>>> failed to load free space cache for block group 5484904972288
> >>>>> [4/7] checking fs roots
> >>>>>
> >>>>> It's currently stuck on that last one.
> >>>>
> >>>> If the fs is pretty large, it can take quite some time.
> >>>>
> >>>> Thanks,
> >>>> Qu
> >>>>
> >>>>>
> >>>>>>
> >>>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Qu
> >>>>>
> >>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 3:25 ` Rosen Penev
@ 2022-04-01 5:05 ` Qu Wenruo
2022-04-01 5:26 ` Rosen Penev
2022-04-01 5:38 ` Rosen Penev
0 siblings, 2 replies; 13+ messages in thread
From: Qu Wenruo @ 2022-04-01 5:05 UTC (permalink / raw)
To: Rosen Penev; +Cc: Qu Wenruo, linux-btrfs
On 2022/4/1 11:25, Rosen Penev wrote:
> On Thu, Mar 31, 2022 at 8:18 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>>
>> On 2022/4/1 11:05, Rosen Penev wrote:
>>> On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2022/4/1 10:48, Rosen Penev wrote:
>>>>> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2022/4/1 08:24, Rosen Penev wrote:
>>>>>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2022/4/1 03:29, Rosen Penev wrote:
>>>>>>>>> A specific folder has files in it. Directly accessing the path works
>>>>>>>>> but ls in the directory returns empty.
>>>>>>>>>
>>>>>>>>> Any way to fix this issue? I believe it happened after a btrfs
>>>>>>>>> replace(failed drive in RAID5) + btrfs balance.
>>>>>>>>
>>>>>>>> Btrfs check please.
>>>>>>> btrfs check --force /dev/sda
>>>>>>
>>>>>> Force is not recommended unless it's your root fs and you don't really
>>>>>> want to run btrfs check on an liveCD.
>>>>> Same result without force and unmounted:
>>>>>
>>>>> btrfs check /dev/sda
>>>>> Opening filesystem to check...
>>>>> Checking filesystem on /dev/sda
>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>>>> [1/7] checking root items
>>>>> [2/7] checking extents
>>>>> [3/7] checking free space cache
>>>>> block group 4885757034496 has wrong amount of free space, free space
>>>>> cache has 286720 block group has 290816
>>>>> failed to load free space cache for block group 4885757034496
>>>>> block group 4898641936384 has wrong amount of free space, free space
>>>>> cache has 36864 block group has 53248
>>>>> failed to load free space cache for block group 4898641936384
>>>>> block group 4953402769408 has wrong amount of free space, free space
>>>>> cache has 262144 block group has 274432
>>>>> failed to load free space cache for block group 4953402769408
>>>>> block group 5478462521344 has wrong amount of free space, free space
>>>>> cache has 716800 block group has 729088
>>>>> failed to load free space cache for block group 5478462521344
>>>>> block group 5484904972288 has wrong amount of free space, free space
>>>>> cache has 811008 block group has 819200
>>>>> failed to load free space cache for block group 5484904972288
>>>>
>>>> Only non-critical free space cache problem, and kernel can detect and
>>>> rebuild them without problem.
>>>>
>>>>> [4/7] checking fs roots
>>>>
>>>> This is the most important part, and it turns no problem at all.
>>>>
>>>> So at least your fs is completely fine.
>>>>
>>>>
>>>> It must be something else causing the problem.
>>>>
>>>> Mind to provide the subvolume id and the inode number (`stat` command
>>>> can return the inode number) of the offending directory?
>>> stat .
>>> File: .
>>> Size: 0 Blocks: 0 IO Block: 4096 directory
>>> Device: 44h/68d Inode: 259 Links: 1
>>> Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
>>> Access: 2022-03-31 19:58:36.577915854 -0700
>>> Modify: 2022-03-13 22:57:55.825138581 -0700
>>> Change: 2022-03-13 22:57:55.825138581 -0700
>>> Birth: 2020-05-16 20:14:31.577476911 -0700
>>>
>>> btrfs subvolume list shows:
>>> ID 259 gen 1084405 top level 5 path Torrents
>>>>
>>>> And some example command output when you can access the files inside the
>>>> directory but `ls -alh` shows nothing?
>>> shows . and .. . That's it.
>>
>> Mind to dump the the following contents?
>> NOTE: this will include file names, feel free to censor filenames if needed:
>>
>> # btrfs ins dump-tree -t 259 <mnt> | grep "(259 " -A8
>>
>> This will dump all info related to inode 259 inside subvolume "Torrents".
> Now it errors :)
>
> btrfs ins dump-tree -t 259 /dev/sda | grep "(259 " -A8
> file tree key (259 ROOT_ITEM 0)
> node 3184169484288 level 2 items 37 free space 456 generation 1084405 owner 259
> node 3184169484288 flags 0x1(WRITTEN) backref revision 1
> fs uuid bfa267c0-df2c-45a6-ad88-9d76b3844326
> chunk uuid 69378187-7a8d-42cc-a28d-a935000d8a94
> key (256 INODE_ITEM 0) block 3184169500672 gen 1084405
> key (1707 INODE_ITEM 0) block 3190334537728 gen 1081752
> key (3045 INODE_ITEM 0) block 3184776314880 gen 1082176
> key (7354 INODE_ITEM 0) block 3184775987200 gen 1082176
> --
> location key (259 INODE_ITEM 0) type DIR
> transid 398 data_len 0 name_len 6
> name: B
> item 9 key (256 DIR_ITEM 2983476959) itemoff 15743 itemsize 35
> location key (257 INODE_ITEM 0) type DIR
> transid 398 data_len 0 name_len 5
> name: A
> item 10 key (256 DIR_ITEM 3061133479) itemoff 15682 itemsize 61
> location key (372987 INODE_ITEM 0) type FILE
> --
> location key (259 INODE_ITEM 0) type DIR
> transid 398 data_len 0 name_len 6
> name: B
> item 19 key (256 DIR_INDEX 5) itemoff 15286 itemsize 36
> location key (260 INODE_ITEM 0) type DIR
> transid 398 data_len 0 name_len 6
> name: C
> item 20 key (256 DIR_INDEX 6) itemoff 15250 itemsize 36
> location key (261 INODE_ITEM 0) type DIR
> --
> item 30 key (259 INODE_ITEM 0) itemoff 13737 itemsize 160
> generation 398 transid 1084405 size 0 nbytes 0
> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> sequence 176 flags 0x0(none)
> atime 1648781916.577915854 (2022-03-31 19:58:36)
> ctime 1647237475.825138581 (2022-03-13 22:57:55)
> mtime 1647237475.825138581 (2022-03-13 22:57:55)
> otime 1589685271.577476911 (2020-05-16 20:14:31)
> item 31 key (259 INODE_REF 256) itemoff 13721 itemsize 16
> index 4 namelen 6 name: B
> item 32 key (260 INODE_ITEM 0) itemoff 13561 itemsize 160
> generation 398 transid 1074067 size 56 nbytes 0
> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> sequence 405 flags 0x0(none)
> atime 1648367720.152037253 (2022-03-27 00:55:20)
> ctime 1646816503.726286540 (2022-03-09 01:01:43)
> mtime 1646816503.726286540 (2022-03-09 01:01:43)
> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> Ignoring transid failure
> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> Ignoring transid failure
> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> Ignoring transid failure
> ERROR: child eb corrupted: parent bytenr=3184315416576 item=4 parent
> level=1 child bytenr=3184317464576 child level=2
Transid mismatch, a big problem.
It can explain the problem.
Maybe kernel can solve it but progs can not really utilize RAID5 parity
to rebuild.
BTW, when you access the directory, no dmesg output?
Thanks,
Qu
>
> Note that name B is the problematic one. I can access A and C just
> fine. Interestingly enough there are many more directories.
>
>>
>> If there is really nothing but the subvolume itself, it may be something
>> else.
>>
>> Thanks,
>> Qu
>>>
>>> The reason I know there are files here is because my torrent client is
>>> currently seeding them. If I change the Catagory (which moves the
>>> files elsewhere), the files show up in the given directory.
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>> [5/7] checking only csums items (without verifying data)
>>>>> [6/7] checking root refs
>>>>> [7/7] checking quota groups skipped (not enabled on this FS)
>>>>> found 2689565679616 bytes used, no error found
>>>>> total csum bytes: 2620609300
>>>>> total tree bytes: 5374935040
>>>>> total fs tree bytes: 1737539584
>>>>> total extent tree bytes: 511115264
>>>>> btree space waste bytes: 889131100
>>>>> file data blocks allocated: 41913072627712
>>>>> referenced 2675025698816
>>>>>
>>>>>>
>>>>>> As sometimes it can report false positive if the fs is not mounted
>>>>>> read-only.
>>>>>>
>>>>>>> Opening filesystem to check...
>>>>>>> Checking filesystem on /dev/sda
>>>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>>>>>> [1/7] checking root items
>>>>>>> [2/7] checking extents
>>>>>>> [3/7] checking free space cache
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 139616845824
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 146059296768
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 3183842689024
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 3184916430848
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 3185990172672
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 3187063914496
>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>> failed to load free space cache for block group 3190318694400
>>>>>>> block group 4885757034496 has wrong amount of free space, free space
>>>>>>> cache has 286720 block group has 290816
>>>>>>> failed to load free space cache for block group 4885757034496
>>>>>>> block group 4898641936384 has wrong amount of free space, free space
>>>>>>> cache has 36864 block group has 53248
>>>>>>> failed to load free space cache for block group 4898641936384
>>>>>>> block group 4953402769408 has wrong amount of free space, free space
>>>>>>> cache has 262144 block group has 274432
>>>>>>> failed to load free space cache for block group 4953402769408
>>>>>>> block group 5478462521344 has wrong amount of free space, free space
>>>>>>> cache has 716800 block group has 729088
>>>>>>> failed to load free space cache for block group 5478462521344
>>>>>>> block group 5484904972288 has wrong amount of free space, free space
>>>>>>> cache has 811008 block group has 819200
>>>>>>> failed to load free space cache for block group 5484904972288
>>>>>>> [4/7] checking fs roots
>>>>>>>
>>>>>>> It's currently stuck on that last one.
>>>>>>
>>>>>> If the fs is pretty large, it can take quite some time.
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Qu
>>>>>>>
>>>>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 5:05 ` Qu Wenruo
@ 2022-04-01 5:26 ` Rosen Penev
2022-04-01 5:43 ` Qu Wenruo
2022-04-01 5:38 ` Rosen Penev
1 sibling, 1 reply; 13+ messages in thread
From: Rosen Penev @ 2022-04-01 5:26 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs
On Thu, Mar 31, 2022 at 10:05 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2022/4/1 11:25, Rosen Penev wrote:
> > On Thu, Mar 31, 2022 at 8:18 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>
> >>
> >>
> >> On 2022/4/1 11:05, Rosen Penev wrote:
> >>> On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 2022/4/1 10:48, Rosen Penev wrote:
> >>>>> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 2022/4/1 08:24, Rosen Penev wrote:
> >>>>>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 2022/4/1 03:29, Rosen Penev wrote:
> >>>>>>>>> A specific folder has files in it. Directly accessing the path works
> >>>>>>>>> but ls in the directory returns empty.
> >>>>>>>>>
> >>>>>>>>> Any way to fix this issue? I believe it happened after a btrfs
> >>>>>>>>> replace(failed drive in RAID5) + btrfs balance.
> >>>>>>>>
> >>>>>>>> Btrfs check please.
> >>>>>>> btrfs check --force /dev/sda
> >>>>>>
> >>>>>> Force is not recommended unless it's your root fs and you don't really
> >>>>>> want to run btrfs check on an liveCD.
> >>>>> Same result without force and unmounted:
> >>>>>
> >>>>> btrfs check /dev/sda
> >>>>> Opening filesystem to check...
> >>>>> Checking filesystem on /dev/sda
> >>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>>>> [1/7] checking root items
> >>>>> [2/7] checking extents
> >>>>> [3/7] checking free space cache
> >>>>> block group 4885757034496 has wrong amount of free space, free space
> >>>>> cache has 286720 block group has 290816
> >>>>> failed to load free space cache for block group 4885757034496
> >>>>> block group 4898641936384 has wrong amount of free space, free space
> >>>>> cache has 36864 block group has 53248
> >>>>> failed to load free space cache for block group 4898641936384
> >>>>> block group 4953402769408 has wrong amount of free space, free space
> >>>>> cache has 262144 block group has 274432
> >>>>> failed to load free space cache for block group 4953402769408
> >>>>> block group 5478462521344 has wrong amount of free space, free space
> >>>>> cache has 716800 block group has 729088
> >>>>> failed to load free space cache for block group 5478462521344
> >>>>> block group 5484904972288 has wrong amount of free space, free space
> >>>>> cache has 811008 block group has 819200
> >>>>> failed to load free space cache for block group 5484904972288
> >>>>
> >>>> Only non-critical free space cache problem, and kernel can detect and
> >>>> rebuild them without problem.
> >>>>
> >>>>> [4/7] checking fs roots
> >>>>
> >>>> This is the most important part, and it turns no problem at all.
> >>>>
> >>>> So at least your fs is completely fine.
> >>>>
> >>>>
> >>>> It must be something else causing the problem.
> >>>>
> >>>> Mind to provide the subvolume id and the inode number (`stat` command
> >>>> can return the inode number) of the offending directory?
> >>> stat .
> >>> File: .
> >>> Size: 0 Blocks: 0 IO Block: 4096 directory
> >>> Device: 44h/68d Inode: 259 Links: 1
> >>> Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
> >>> Access: 2022-03-31 19:58:36.577915854 -0700
> >>> Modify: 2022-03-13 22:57:55.825138581 -0700
> >>> Change: 2022-03-13 22:57:55.825138581 -0700
> >>> Birth: 2020-05-16 20:14:31.577476911 -0700
> >>>
> >>> btrfs subvolume list shows:
> >>> ID 259 gen 1084405 top level 5 path Torrents
> >>>>
> >>>> And some example command output when you can access the files inside the
> >>>> directory but `ls -alh` shows nothing?
> >>> shows . and .. . That's it.
> >>
> >> Mind to dump the the following contents?
> >> NOTE: this will include file names, feel free to censor filenames if needed:
> >>
> >> # btrfs ins dump-tree -t 259 <mnt> | grep "(259 " -A8
> >>
> >> This will dump all info related to inode 259 inside subvolume "Torrents".
> > Now it errors :)
> >
> > btrfs ins dump-tree -t 259 /dev/sda | grep "(259 " -A8
> > file tree key (259 ROOT_ITEM 0)
> > node 3184169484288 level 2 items 37 free space 456 generation 1084405 owner 259
> > node 3184169484288 flags 0x1(WRITTEN) backref revision 1
> > fs uuid bfa267c0-df2c-45a6-ad88-9d76b3844326
> > chunk uuid 69378187-7a8d-42cc-a28d-a935000d8a94
> > key (256 INODE_ITEM 0) block 3184169500672 gen 1084405
> > key (1707 INODE_ITEM 0) block 3190334537728 gen 1081752
> > key (3045 INODE_ITEM 0) block 3184776314880 gen 1082176
> > key (7354 INODE_ITEM 0) block 3184775987200 gen 1082176
> > --
> > location key (259 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 6
> > name: B
> > item 9 key (256 DIR_ITEM 2983476959) itemoff 15743 itemsize 35
> > location key (257 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 5
> > name: A
> > item 10 key (256 DIR_ITEM 3061133479) itemoff 15682 itemsize 61
> > location key (372987 INODE_ITEM 0) type FILE
> > --
> > location key (259 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 6
> > name: B
> > item 19 key (256 DIR_INDEX 5) itemoff 15286 itemsize 36
> > location key (260 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 6
> > name: C
> > item 20 key (256 DIR_INDEX 6) itemoff 15250 itemsize 36
> > location key (261 INODE_ITEM 0) type DIR
> > --
> > item 30 key (259 INODE_ITEM 0) itemoff 13737 itemsize 160
> > generation 398 transid 1084405 size 0 nbytes 0
> > block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> > sequence 176 flags 0x0(none)
> > atime 1648781916.577915854 (2022-03-31 19:58:36)
> > ctime 1647237475.825138581 (2022-03-13 22:57:55)
> > mtime 1647237475.825138581 (2022-03-13 22:57:55)
> > otime 1589685271.577476911 (2020-05-16 20:14:31)
> > item 31 key (259 INODE_REF 256) itemoff 13721 itemsize 16
> > index 4 namelen 6 name: B
> > item 32 key (260 INODE_ITEM 0) itemoff 13561 itemsize 160
> > generation 398 transid 1074067 size 56 nbytes 0
> > block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> > sequence 405 flags 0x0(none)
> > atime 1648367720.152037253 (2022-03-27 00:55:20)
> > ctime 1646816503.726286540 (2022-03-09 01:01:43)
> > mtime 1646816503.726286540 (2022-03-09 01:01:43)
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > Ignoring transid failure
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > Ignoring transid failure
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > Ignoring transid failure
> > ERROR: child eb corrupted: parent bytenr=3184315416576 item=4 parent
> > level=1 child bytenr=3184317464576 child level=2
>
> Transid mismatch, a big problem.
> It can explain the problem.
>
> Maybe kernel can solve it but progs can not really utilize RAID5 parity
> to rebuild.
>
> BTW, when you access the directory, no dmesg output?
Nope. I get dmesg output when accessing a specific file (QuasselIRC
database). Maybe that's what the output was referring to.
[ 4989.026269] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 99, gen 0
[ 4989.046520] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 4989.046742] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 4989.046753] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 100, gen 0
[ 4989.046963] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 5197.247306] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 5197.247326] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 101, gen 0
[ 5197.247648] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 5725.000447] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 5725.000469] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 102, gen 0
[ 5725.000835] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 5725.001099] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 5725.001115] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 103, gen 0
[ 5725.001369] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 5725.002310] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 5725.002326] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 104, gen 0
[ 5725.002602] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 5750.381840] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 5750.381860] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 105, gen 0
[ 5750.382338] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 5754.388921] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
[ 5754.388941] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 106, gen 0
[ 5754.389262] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
[ 9098.137439] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
[ 9098.137459] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 107, gen 0
[ 9098.160482] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
[ 9102.465457] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
[ 9102.465478] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 108, gen 0
[ 9102.465831] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
[ 9102.466118] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
[ 9102.466129] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 109, gen 0
[ 9102.466368] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
[ 9102.466555] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
[ 9102.466565] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
0, flush 0, corrupt 110, gen 0
[ 9102.466765] BTRFS warning (device sda): csum failed root 1094 ino
157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
>
> Thanks,
> Qu
>
> >
> > Note that name B is the problematic one. I can access A and C just
> > fine. Interestingly enough there are many more directories.
> >
> >>
> >> If there is really nothing but the subvolume itself, it may be something
> >> else.
> >>
> >> Thanks,
> >> Qu
> >>>
> >>> The reason I know there are files here is because my torrent client is
> >>> currently seeding them. If I change the Catagory (which moves the
> >>> files elsewhere), the files show up in the given directory.
> >>>>
> >>>> Thanks,
> >>>> Qu
> >>>>
> >>>>> [5/7] checking only csums items (without verifying data)
> >>>>> [6/7] checking root refs
> >>>>> [7/7] checking quota groups skipped (not enabled on this FS)
> >>>>> found 2689565679616 bytes used, no error found
> >>>>> total csum bytes: 2620609300
> >>>>> total tree bytes: 5374935040
> >>>>> total fs tree bytes: 1737539584
> >>>>> total extent tree bytes: 511115264
> >>>>> btree space waste bytes: 889131100
> >>>>> file data blocks allocated: 41913072627712
> >>>>> referenced 2675025698816
> >>>>>
> >>>>>>
> >>>>>> As sometimes it can report false positive if the fs is not mounted
> >>>>>> read-only.
> >>>>>>
> >>>>>>> Opening filesystem to check...
> >>>>>>> Checking filesystem on /dev/sda
> >>>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>>>>>> [1/7] checking root items
> >>>>>>> [2/7] checking extents
> >>>>>>> [3/7] checking free space cache
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 139616845824
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 146059296768
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3183842689024
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3184916430848
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3185990172672
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3187063914496
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3190318694400
> >>>>>>> block group 4885757034496 has wrong amount of free space, free space
> >>>>>>> cache has 286720 block group has 290816
> >>>>>>> failed to load free space cache for block group 4885757034496
> >>>>>>> block group 4898641936384 has wrong amount of free space, free space
> >>>>>>> cache has 36864 block group has 53248
> >>>>>>> failed to load free space cache for block group 4898641936384
> >>>>>>> block group 4953402769408 has wrong amount of free space, free space
> >>>>>>> cache has 262144 block group has 274432
> >>>>>>> failed to load free space cache for block group 4953402769408
> >>>>>>> block group 5478462521344 has wrong amount of free space, free space
> >>>>>>> cache has 716800 block group has 729088
> >>>>>>> failed to load free space cache for block group 5478462521344
> >>>>>>> block group 5484904972288 has wrong amount of free space, free space
> >>>>>>> cache has 811008 block group has 819200
> >>>>>>> failed to load free space cache for block group 5484904972288
> >>>>>>> [4/7] checking fs roots
> >>>>>>>
> >>>>>>> It's currently stuck on that last one.
> >>>>>>
> >>>>>> If the fs is pretty large, it can take quite some time.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Qu
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Qu
> >>>>>>>
> >>>>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 5:05 ` Qu Wenruo
2022-04-01 5:26 ` Rosen Penev
@ 2022-04-01 5:38 ` Rosen Penev
1 sibling, 0 replies; 13+ messages in thread
From: Rosen Penev @ 2022-04-01 5:38 UTC (permalink / raw)
To: Qu Wenruo; +Cc: Qu Wenruo, linux-btrfs
On Thu, Mar 31, 2022 at 10:05 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>
>
>
> On 2022/4/1 11:25, Rosen Penev wrote:
> > On Thu, Mar 31, 2022 at 8:18 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>
> >>
> >>
> >> On 2022/4/1 11:05, Rosen Penev wrote:
> >>> On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On 2022/4/1 10:48, Rosen Penev wrote:
> >>>>> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On 2022/4/1 08:24, Rosen Penev wrote:
> >>>>>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 2022/4/1 03:29, Rosen Penev wrote:
> >>>>>>>>> A specific folder has files in it. Directly accessing the path works
> >>>>>>>>> but ls in the directory returns empty.
> >>>>>>>>>
> >>>>>>>>> Any way to fix this issue? I believe it happened after a btrfs
> >>>>>>>>> replace(failed drive in RAID5) + btrfs balance.
> >>>>>>>>
> >>>>>>>> Btrfs check please.
> >>>>>>> btrfs check --force /dev/sda
> >>>>>>
> >>>>>> Force is not recommended unless it's your root fs and you don't really
> >>>>>> want to run btrfs check on an liveCD.
> >>>>> Same result without force and unmounted:
> >>>>>
> >>>>> btrfs check /dev/sda
> >>>>> Opening filesystem to check...
> >>>>> Checking filesystem on /dev/sda
> >>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>>>> [1/7] checking root items
> >>>>> [2/7] checking extents
> >>>>> [3/7] checking free space cache
> >>>>> block group 4885757034496 has wrong amount of free space, free space
> >>>>> cache has 286720 block group has 290816
> >>>>> failed to load free space cache for block group 4885757034496
> >>>>> block group 4898641936384 has wrong amount of free space, free space
> >>>>> cache has 36864 block group has 53248
> >>>>> failed to load free space cache for block group 4898641936384
> >>>>> block group 4953402769408 has wrong amount of free space, free space
> >>>>> cache has 262144 block group has 274432
> >>>>> failed to load free space cache for block group 4953402769408
> >>>>> block group 5478462521344 has wrong amount of free space, free space
> >>>>> cache has 716800 block group has 729088
> >>>>> failed to load free space cache for block group 5478462521344
> >>>>> block group 5484904972288 has wrong amount of free space, free space
> >>>>> cache has 811008 block group has 819200
> >>>>> failed to load free space cache for block group 5484904972288
> >>>>
> >>>> Only non-critical free space cache problem, and kernel can detect and
> >>>> rebuild them without problem.
> >>>>
> >>>>> [4/7] checking fs roots
> >>>>
> >>>> This is the most important part, and it turns no problem at all.
> >>>>
> >>>> So at least your fs is completely fine.
> >>>>
> >>>>
> >>>> It must be something else causing the problem.
> >>>>
> >>>> Mind to provide the subvolume id and the inode number (`stat` command
> >>>> can return the inode number) of the offending directory?
> >>> stat .
> >>> File: .
> >>> Size: 0 Blocks: 0 IO Block: 4096 directory
> >>> Device: 44h/68d Inode: 259 Links: 1
> >>> Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
> >>> Access: 2022-03-31 19:58:36.577915854 -0700
> >>> Modify: 2022-03-13 22:57:55.825138581 -0700
> >>> Change: 2022-03-13 22:57:55.825138581 -0700
> >>> Birth: 2020-05-16 20:14:31.577476911 -0700
> >>>
> >>> btrfs subvolume list shows:
> >>> ID 259 gen 1084405 top level 5 path Torrents
> >>>>
> >>>> And some example command output when you can access the files inside the
> >>>> directory but `ls -alh` shows nothing?
> >>> shows . and .. . That's it.
> >>
> >> Mind to dump the the following contents?
> >> NOTE: this will include file names, feel free to censor filenames if needed:
> >>
> >> # btrfs ins dump-tree -t 259 <mnt> | grep "(259 " -A8
> >>
> >> This will dump all info related to inode 259 inside subvolume "Torrents".
> > Now it errors :)
> >
> > btrfs ins dump-tree -t 259 /dev/sda | grep "(259 " -A8
> > file tree key (259 ROOT_ITEM 0)
> > node 3184169484288 level 2 items 37 free space 456 generation 1084405 owner 259
> > node 3184169484288 flags 0x1(WRITTEN) backref revision 1
> > fs uuid bfa267c0-df2c-45a6-ad88-9d76b3844326
> > chunk uuid 69378187-7a8d-42cc-a28d-a935000d8a94
> > key (256 INODE_ITEM 0) block 3184169500672 gen 1084405
> > key (1707 INODE_ITEM 0) block 3190334537728 gen 1081752
> > key (3045 INODE_ITEM 0) block 3184776314880 gen 1082176
> > key (7354 INODE_ITEM 0) block 3184775987200 gen 1082176
> > --
> > location key (259 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 6
> > name: B
> > item 9 key (256 DIR_ITEM 2983476959) itemoff 15743 itemsize 35
> > location key (257 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 5
> > name: A
> > item 10 key (256 DIR_ITEM 3061133479) itemoff 15682 itemsize 61
> > location key (372987 INODE_ITEM 0) type FILE
> > --
> > location key (259 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 6
> > name: B
> > item 19 key (256 DIR_INDEX 5) itemoff 15286 itemsize 36
> > location key (260 INODE_ITEM 0) type DIR
> > transid 398 data_len 0 name_len 6
> > name: C
> > item 20 key (256 DIR_INDEX 6) itemoff 15250 itemsize 36
> > location key (261 INODE_ITEM 0) type DIR
> > --
> > item 30 key (259 INODE_ITEM 0) itemoff 13737 itemsize 160
> > generation 398 transid 1084405 size 0 nbytes 0
> > block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> > sequence 176 flags 0x0(none)
> > atime 1648781916.577915854 (2022-03-31 19:58:36)
> > ctime 1647237475.825138581 (2022-03-13 22:57:55)
> > mtime 1647237475.825138581 (2022-03-13 22:57:55)
> > otime 1589685271.577476911 (2020-05-16 20:14:31)
> > item 31 key (259 INODE_REF 256) itemoff 13721 itemsize 16
> > index 4 namelen 6 name: B
> > item 32 key (260 INODE_ITEM 0) itemoff 13561 itemsize 160
> > generation 398 transid 1074067 size 56 nbytes 0
> > block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> > sequence 405 flags 0x0(none)
> > atime 1648367720.152037253 (2022-03-27 00:55:20)
> > ctime 1646816503.726286540 (2022-03-09 01:01:43)
> > mtime 1646816503.726286540 (2022-03-09 01:01:43)
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
> > Ignoring transid failure
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
> > Ignoring transid failure
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
> > Ignoring transid failure
> > ERROR: child eb corrupted: parent bytenr=3184315416576 item=4 parent
> > level=1 child bytenr=3184317464576 child level=2
>
> Transid mismatch, a big problem.
> It can explain the problem.
Alright. I actually found my files. Turns out, there was some
configuration change in LinuxServer.io's qBittorrent docker container
or maybe an upstream one that resulted in them being placed within the
docker volume instead of the actual directory. I'll probably be
rebuilding my setup...
Sorry for the noise.
>
> Maybe kernel can solve it but progs can not really utilize RAID5 parity
> to rebuild.
>
> BTW, when you access the directory, no dmesg output?
>
> Thanks,
> Qu
>
> >
> > Note that name B is the problematic one. I can access A and C just
> > fine. Interestingly enough there are many more directories.
> >
> >>
> >> If there is really nothing but the subvolume itself, it may be something
> >> else.
> >>
> >> Thanks,
> >> Qu
> >>>
> >>> The reason I know there are files here is because my torrent client is
> >>> currently seeding them. If I change the Catagory (which moves the
> >>> files elsewhere), the files show up in the given directory.
> >>>>
> >>>> Thanks,
> >>>> Qu
> >>>>
> >>>>> [5/7] checking only csums items (without verifying data)
> >>>>> [6/7] checking root refs
> >>>>> [7/7] checking quota groups skipped (not enabled on this FS)
> >>>>> found 2689565679616 bytes used, no error found
> >>>>> total csum bytes: 2620609300
> >>>>> total tree bytes: 5374935040
> >>>>> total fs tree bytes: 1737539584
> >>>>> total extent tree bytes: 511115264
> >>>>> btree space waste bytes: 889131100
> >>>>> file data blocks allocated: 41913072627712
> >>>>> referenced 2675025698816
> >>>>>
> >>>>>>
> >>>>>> As sometimes it can report false positive if the fs is not mounted
> >>>>>> read-only.
> >>>>>>
> >>>>>>> Opening filesystem to check...
> >>>>>>> Checking filesystem on /dev/sda
> >>>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
> >>>>>>> [1/7] checking root items
> >>>>>>> [2/7] checking extents
> >>>>>>> [3/7] checking free space cache
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 139616845824
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 146059296768
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3183842689024
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3184916430848
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3185990172672
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3187063914496
> >>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
> >>>>>>> failed to load free space cache for block group 3190318694400
> >>>>>>> block group 4885757034496 has wrong amount of free space, free space
> >>>>>>> cache has 286720 block group has 290816
> >>>>>>> failed to load free space cache for block group 4885757034496
> >>>>>>> block group 4898641936384 has wrong amount of free space, free space
> >>>>>>> cache has 36864 block group has 53248
> >>>>>>> failed to load free space cache for block group 4898641936384
> >>>>>>> block group 4953402769408 has wrong amount of free space, free space
> >>>>>>> cache has 262144 block group has 274432
> >>>>>>> failed to load free space cache for block group 4953402769408
> >>>>>>> block group 5478462521344 has wrong amount of free space, free space
> >>>>>>> cache has 716800 block group has 729088
> >>>>>>> failed to load free space cache for block group 5478462521344
> >>>>>>> block group 5484904972288 has wrong amount of free space, free space
> >>>>>>> cache has 811008 block group has 819200
> >>>>>>> failed to load free space cache for block group 5484904972288
> >>>>>>> [4/7] checking fs roots
> >>>>>>>
> >>>>>>> It's currently stuck on that last one.
> >>>>>>
> >>>>>> If the fs is pretty large, it can take quite some time.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Qu
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Qu
> >>>>>>>
> >>>>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs volume can't see files in folder
2022-04-01 5:26 ` Rosen Penev
@ 2022-04-01 5:43 ` Qu Wenruo
0 siblings, 0 replies; 13+ messages in thread
From: Qu Wenruo @ 2022-04-01 5:43 UTC (permalink / raw)
To: Rosen Penev; +Cc: Qu Wenruo, linux-btrfs
On 2022/4/1 13:26, Rosen Penev wrote:
> On Thu, Mar 31, 2022 at 10:05 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>
>>
>>
>> On 2022/4/1 11:25, Rosen Penev wrote:
>>> On Thu, Mar 31, 2022 at 8:18 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>
>>>>
>>>>
>>>> On 2022/4/1 11:05, Rosen Penev wrote:
>>>>> On Thu, Mar 31, 2022 at 7:53 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2022/4/1 10:48, Rosen Penev wrote:
>>>>>>> On Thu, Mar 31, 2022 at 5:59 PM Qu Wenruo <wqu@suse.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2022/4/1 08:24, Rosen Penev wrote:
>>>>>>>>> On Thu, Mar 31, 2022 at 4:40 PM Qu Wenruo <quwenruo.btrfs@gmx.com> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2022/4/1 03:29, Rosen Penev wrote:
>>>>>>>>>>> A specific folder has files in it. Directly accessing the path works
>>>>>>>>>>> but ls in the directory returns empty.
>>>>>>>>>>>
>>>>>>>>>>> Any way to fix this issue? I believe it happened after a btrfs
>>>>>>>>>>> replace(failed drive in RAID5) + btrfs balance.
>>>>>>>>>>
>>>>>>>>>> Btrfs check please.
>>>>>>>>> btrfs check --force /dev/sda
>>>>>>>>
>>>>>>>> Force is not recommended unless it's your root fs and you don't really
>>>>>>>> want to run btrfs check on an liveCD.
>>>>>>> Same result without force and unmounted:
>>>>>>>
>>>>>>> btrfs check /dev/sda
>>>>>>> Opening filesystem to check...
>>>>>>> Checking filesystem on /dev/sda
>>>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>>>>>> [1/7] checking root items
>>>>>>> [2/7] checking extents
>>>>>>> [3/7] checking free space cache
>>>>>>> block group 4885757034496 has wrong amount of free space, free space
>>>>>>> cache has 286720 block group has 290816
>>>>>>> failed to load free space cache for block group 4885757034496
>>>>>>> block group 4898641936384 has wrong amount of free space, free space
>>>>>>> cache has 36864 block group has 53248
>>>>>>> failed to load free space cache for block group 4898641936384
>>>>>>> block group 4953402769408 has wrong amount of free space, free space
>>>>>>> cache has 262144 block group has 274432
>>>>>>> failed to load free space cache for block group 4953402769408
>>>>>>> block group 5478462521344 has wrong amount of free space, free space
>>>>>>> cache has 716800 block group has 729088
>>>>>>> failed to load free space cache for block group 5478462521344
>>>>>>> block group 5484904972288 has wrong amount of free space, free space
>>>>>>> cache has 811008 block group has 819200
>>>>>>> failed to load free space cache for block group 5484904972288
>>>>>>
>>>>>> Only non-critical free space cache problem, and kernel can detect and
>>>>>> rebuild them without problem.
>>>>>>
>>>>>>> [4/7] checking fs roots
>>>>>>
>>>>>> This is the most important part, and it turns no problem at all.
>>>>>>
>>>>>> So at least your fs is completely fine.
>>>>>>
>>>>>>
>>>>>> It must be something else causing the problem.
>>>>>>
>>>>>> Mind to provide the subvolume id and the inode number (`stat` command
>>>>>> can return the inode number) of the offending directory?
>>>>> stat .
>>>>> File: .
>>>>> Size: 0 Blocks: 0 IO Block: 4096 directory
>>>>> Device: 44h/68d Inode: 259 Links: 1
>>>>> Access: (0755/drwxr-xr-x) Uid: ( 1000/ mangix) Gid: ( 1000/ mangix)
>>>>> Access: 2022-03-31 19:58:36.577915854 -0700
>>>>> Modify: 2022-03-13 22:57:55.825138581 -0700
>>>>> Change: 2022-03-13 22:57:55.825138581 -0700
>>>>> Birth: 2020-05-16 20:14:31.577476911 -0700
>>>>>
>>>>> btrfs subvolume list shows:
>>>>> ID 259 gen 1084405 top level 5 path Torrents
>>>>>>
>>>>>> And some example command output when you can access the files inside the
>>>>>> directory but `ls -alh` shows nothing?
>>>>> shows . and .. . That's it.
>>>>
>>>> Mind to dump the the following contents?
>>>> NOTE: this will include file names, feel free to censor filenames if needed:
>>>>
>>>> # btrfs ins dump-tree -t 259 <mnt> | grep "(259 " -A8
>>>>
>>>> This will dump all info related to inode 259 inside subvolume "Torrents".
>>> Now it errors :)
>>>
>>> btrfs ins dump-tree -t 259 /dev/sda | grep "(259 " -A8
>>> file tree key (259 ROOT_ITEM 0)
>>> node 3184169484288 level 2 items 37 free space 456 generation 1084405 owner 259
>>> node 3184169484288 flags 0x1(WRITTEN) backref revision 1
>>> fs uuid bfa267c0-df2c-45a6-ad88-9d76b3844326
>>> chunk uuid 69378187-7a8d-42cc-a28d-a935000d8a94
>>> key (256 INODE_ITEM 0) block 3184169500672 gen 1084405
>>> key (1707 INODE_ITEM 0) block 3190334537728 gen 1081752
>>> key (3045 INODE_ITEM 0) block 3184776314880 gen 1082176
>>> key (7354 INODE_ITEM 0) block 3184775987200 gen 1082176
>>> --
>>> location key (259 INODE_ITEM 0) type DIR
>>> transid 398 data_len 0 name_len 6
>>> name: B
>>> item 9 key (256 DIR_ITEM 2983476959) itemoff 15743 itemsize 35
>>> location key (257 INODE_ITEM 0) type DIR
>>> transid 398 data_len 0 name_len 5
>>> name: A
>>> item 10 key (256 DIR_ITEM 3061133479) itemoff 15682 itemsize 61
>>> location key (372987 INODE_ITEM 0) type FILE
>>> --
>>> location key (259 INODE_ITEM 0) type DIR
>>> transid 398 data_len 0 name_len 6
>>> name: B
>>> item 19 key (256 DIR_INDEX 5) itemoff 15286 itemsize 36
>>> location key (260 INODE_ITEM 0) type DIR
>>> transid 398 data_len 0 name_len 6
>>> name: C
>>> item 20 key (256 DIR_INDEX 6) itemoff 15250 itemsize 36
>>> location key (261 INODE_ITEM 0) type DIR
>>> --
>>> item 30 key (259 INODE_ITEM 0) itemoff 13737 itemsize 160
>>> generation 398 transid 1084405 size 0 nbytes 0
>>> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
>>> sequence 176 flags 0x0(none)
>>> atime 1648781916.577915854 (2022-03-31 19:58:36)
>>> ctime 1647237475.825138581 (2022-03-13 22:57:55)
>>> mtime 1647237475.825138581 (2022-03-13 22:57:55)
>>> otime 1589685271.577476911 (2020-05-16 20:14:31)
>>> item 31 key (259 INODE_REF 256) itemoff 13721 itemsize 16
>>> index 4 namelen 6 name: B
>>> item 32 key (260 INODE_ITEM 0) itemoff 13561 itemsize 160
>>> generation 398 transid 1074067 size 56 nbytes 0
>>> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
>>> sequence 405 flags 0x0(none)
>>> atime 1648367720.152037253 (2022-03-27 00:55:20)
>>> ctime 1646816503.726286540 (2022-03-09 01:01:43)
>>> mtime 1646816503.726286540 (2022-03-09 01:01:43)
>>> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
>>> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
>>> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
>>> parent transid verify failed on 3184315432960 wanted 1084439 found 1084442
>>> Ignoring transid failure
>>> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
>>> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
>>> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
>>> parent transid verify failed on 3184315498496 wanted 1084439 found 1084442
>>> Ignoring transid failure
>>> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
>>> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
>>> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
>>> parent transid verify failed on 3184317464576 wanted 1084439 found 1084443
>>> Ignoring transid failure
>>> ERROR: child eb corrupted: parent bytenr=3184315416576 item=4 parent
>>> level=1 child bytenr=3184317464576 child level=2
>>
>> Transid mismatch, a big problem.
>> It can explain the problem.
>>
>> Maybe kernel can solve it but progs can not really utilize RAID5 parity
>> to rebuild.
>>
>> BTW, when you access the directory, no dmesg output?
> Nope. I get dmesg output when accessing a specific file (QuasselIRC
> database). Maybe that's what the output was referring to.
>
> [ 4989.026269] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 99, gen 0
> [ 4989.046520] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 4989.046742] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 4989.046753] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 100, gen 0
> [ 4989.046963] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 5197.247306] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 5197.247326] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 101, gen 0
> [ 5197.247648] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 5725.000447] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 5725.000469] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 102, gen 0
> [ 5725.000835] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 5725.001099] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 5725.001115] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 103, gen 0
> [ 5725.001369] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 5725.002310] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 5725.002326] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 104, gen 0
> [ 5725.002602] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 5750.381840] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 5750.381860] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 105, gen 0
> [ 5750.382338] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 5754.388921] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 1
> [ 5754.388941] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 106, gen 0
> [ 5754.389262] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 246456320 csum 0x4fcb51d2 expected csum 0x981271e6 mirror 2
> [ 9098.137439] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
> [ 9098.137459] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 107, gen 0
> [ 9098.160482] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
> [ 9102.465457] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
> [ 9102.465478] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 108, gen 0
> [ 9102.465831] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
> [ 9102.466118] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
> [ 9102.466129] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 109, gen 0
> [ 9102.466368] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
> [ 9102.466555] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 1
> [ 9102.466565] BTRFS error (device sda): bdev /dev/sdb errs: wr 0, rd
> 0, flush 0, corrupt 110, gen 0
> [ 9102.466765] BTRFS warning (device sda): csum failed root 1094 ino
> 157635 off 150896640 csum 0x209b44e3 expected csum 0x06aeaa69 mirror 2
Just data corruption, not a big deal.
One thing check is, the previous "btrfs ins dump-tree" should be called
on an unmounted fs, or the transid can be a false alert if the mounted
fs choose to commit a transaction.
And since you have figured out why your files are "missing", and have a
full btrfs check run, it at least proves your fs is fine (at least for
metadata).
The corrupted data may be a symptom of RAID56 write hole.
In the future, it's strongly recommended to run scrub after each
unexpected powerloss/forced shutdown to reduce the impact of RAID56
write hole.
Thanks,
Qu
>>
>> Thanks,
>> Qu
>>
>>>
>>> Note that name B is the problematic one. I can access A and C just
>>> fine. Interestingly enough there are many more directories.
>>>
>>>>
>>>> If there is really nothing but the subvolume itself, it may be something
>>>> else.
>>>>
>>>> Thanks,
>>>> Qu
>>>>>
>>>>> The reason I know there are files here is because my torrent client is
>>>>> currently seeding them. If I change the Catagory (which moves the
>>>>> files elsewhere), the files show up in the given directory.
>>>>>>
>>>>>> Thanks,
>>>>>> Qu
>>>>>>
>>>>>>> [5/7] checking only csums items (without verifying data)
>>>>>>> [6/7] checking root refs
>>>>>>> [7/7] checking quota groups skipped (not enabled on this FS)
>>>>>>> found 2689565679616 bytes used, no error found
>>>>>>> total csum bytes: 2620609300
>>>>>>> total tree bytes: 5374935040
>>>>>>> total fs tree bytes: 1737539584
>>>>>>> total extent tree bytes: 511115264
>>>>>>> btree space waste bytes: 889131100
>>>>>>> file data blocks allocated: 41913072627712
>>>>>>> referenced 2675025698816
>>>>>>>
>>>>>>>>
>>>>>>>> As sometimes it can report false positive if the fs is not mounted
>>>>>>>> read-only.
>>>>>>>>
>>>>>>>>> Opening filesystem to check...
>>>>>>>>> Checking filesystem on /dev/sda
>>>>>>>>> UUID: bfa267c0-df2c-45a6-ad88-9d76b3844326
>>>>>>>>> [1/7] checking root items
>>>>>>>>> [2/7] checking extents
>>>>>>>>> [3/7] checking free space cache
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 139616845824
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 146059296768
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 3183842689024
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 3184916430848
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 3185990172672
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 3187063914496
>>>>>>>>> btrfs: space cache generation (1084391) does not match inode (1084389)
>>>>>>>>> failed to load free space cache for block group 3190318694400
>>>>>>>>> block group 4885757034496 has wrong amount of free space, free space
>>>>>>>>> cache has 286720 block group has 290816
>>>>>>>>> failed to load free space cache for block group 4885757034496
>>>>>>>>> block group 4898641936384 has wrong amount of free space, free space
>>>>>>>>> cache has 36864 block group has 53248
>>>>>>>>> failed to load free space cache for block group 4898641936384
>>>>>>>>> block group 4953402769408 has wrong amount of free space, free space
>>>>>>>>> cache has 262144 block group has 274432
>>>>>>>>> failed to load free space cache for block group 4953402769408
>>>>>>>>> block group 5478462521344 has wrong amount of free space, free space
>>>>>>>>> cache has 716800 block group has 729088
>>>>>>>>> failed to load free space cache for block group 5478462521344
>>>>>>>>> block group 5484904972288 has wrong amount of free space, free space
>>>>>>>>> cache has 811008 block group has 819200
>>>>>>>>> failed to load free space cache for block group 5484904972288
>>>>>>>>> [4/7] checking fs roots
>>>>>>>>>
>>>>>>>>> It's currently stuck on that last one.
>>>>>>>>
>>>>>>>> If the fs is pretty large, it can take quite some time.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Qu
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It looks like an DIR_ITEM/DIR_INDEX corruption.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Qu
>>>>>>>>>
>>>>>>>>
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2022-04-01 5:43 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-31 19:29 btrfs volume can't see files in folder Rosen Penev
2022-03-31 23:39 ` Qu Wenruo
2022-04-01 0:24 ` Rosen Penev
2022-04-01 0:59 ` Qu Wenruo
2022-04-01 2:48 ` Rosen Penev
2022-04-01 2:53 ` Qu Wenruo
2022-04-01 3:05 ` Rosen Penev
2022-04-01 3:18 ` Qu Wenruo
2022-04-01 3:25 ` Rosen Penev
2022-04-01 5:05 ` Qu Wenruo
2022-04-01 5:26 ` Rosen Penev
2022-04-01 5:43 ` Qu Wenruo
2022-04-01 5:38 ` Rosen Penev
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.