btrfsck segmentation fault + trace

btrfsck segmentation fault + trace

[John Wyzer]

I tried 2.6.35-rc3 and got error messages during the boot process about not being able
to dirty inodes and no space left on device.

2.6.34.y runs fine and df -h shows
rootfs                296G  255G   39G  87% /

I reverted back to 2.6.34 and gave btrfsck a try resulting in a segmentation fault.

# btrfsck /dev/mapper/root 
parent transid verify failed on 217995780096 wanted 155238 found 155234
btrfsck: disk-io.c:416: find_and_setup_root: Assertion `!(!root->node)' failed.
Aborted (core dumped)


Core was generated by `btrfsck /dev/mapper/root'.
Program terminated with signal 6, Aborted.
#0  0xffffe424 in __kernel_vsyscall ()
(gdb) bt
#0  0xffffe424 in __kernel_vsyscall ()
#1  0xb76c2751 in *__GI_raise (sig=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
#2  0xb76c5b82 in *__GI_abort () at abort.c:92
#3  0xb76bb8b8 in *__GI___assert_fail (assertion=0x8062dc7 "!(!root->node)", file=0x8062d2d "disk-io.c", line=416, 
    function=0x8062fe0 "find_and_setup_root") at assert.c:81
#4  0x080525bb in find_and_setup_root (tree_root=0x9852008, fs_info=<value optimized out>, objectid=2, root=0x9852178)
    at disk-io.c:416
#5  0x08052dcd in open_ctree_fd (fp=3, path=0xbfd984f9 "/dev/mapper/root", sb_bytenr=0, writes=0) at disk-io.c:740
#6  0x0805303d in open_ctree (filename=0xbfd984f9 "/dev/mapper/root", sb_bytenr=0, writes=0) at disk-io.c:587
#7  0x0804d36b in main (ac=2, av=0xbfd98344) at btrfsck.c:2824
(gdb) 

Hope that helps in any way :-)
Best regards.

Re: btrfsck segmentation fault + trace

[Chris Mason]

On Wed, Jun 16, 2010 at 05:52:40PM +0200, John Wyzer wrote:
> I tried 2.6.35-rc3 and got error messages during the boot process about not being able
> to dirty inodes and no space left on device.

Is 2.6.34 working normally?

> 
> 2.6.34.y runs fine and df -h shows
> rootfs                296G  255G   39G  87% /
> 
> I reverted back to 2.6.34 and gave btrfsck a try resulting in a segmentation fault.
> 
> # btrfsck /dev/mapper/root 
> parent transid verify failed on 217995780096 wanted 155238 found 155234
> btrfsck: disk-io.c:416: find_and_setup_root: Assertion `!(!root->node)' failed.
> Aborted (core dumped)

This shows a bad metadata block, basically a missed write.  It sounds
like something has gone wrong with this system.

-chris

Re: btrfsck segmentation fault + trace

[John Wyzer]

On 16/06/10 18:31, Chris Mason wrote:
> On Wed, Jun 16, 2010 at 05:52:40PM +0200, John Wyzer wrote:
>> I tried 2.6.35-rc3 and got error messages during the boot process about not being able
>> to dirty inodes and no space left on device.
> 
> Is 2.6.34 working normally?


Yes. I can boot with 2.6.34.y and everything works fine. (Actually,
before trying 2.6.45-rc3, I had an uptime of two weeks on this laptop.
Now, I'm writing this email on 2.6.34.y.)

Re: btrfsck segmentation fault + trace

[Chris Mason]

On Wed, Jun 16, 2010 at 06:35:25PM +0200, John Wyzer wrote:
> On 16/06/10 18:31, Chris Mason wrote:
> > On Wed, Jun 16, 2010 at 05:52:40PM +0200, John Wyzer wrote:
> >> I tried 2.6.35-rc3 and got error messages during the boot process about not being able
> >> to dirty inodes and no space left on device.
> > 
> > Is 2.6.34 working normally?
> 
> 
> Yes. I can boot with 2.6.34.y and everything works fine. (Actually,
> before trying 2.6.45-rc3, I had an uptime of two weeks on this laptop.
> Now, I'm writing this email on 2.6.34.y.)

But every time you boot 2.6.35 you get errors?  Would it be possible to
save the console output (netconsole works well)

-chris

Re: btrfsck segmentation fault + trace

[John Wyzer]

On 16/06/10 19:29, Chris Mason wrote:
> On Wed, Jun 16, 2010 at 06:35:25PM +0200, John Wyzer wrote:
>>> Is 2.6.34 working normally?
>>
>> Yes. I can boot with 2.6.34.y and everything works fine. (Actually,
>> before trying 2.6.45-rc3, I had an uptime of two weeks on this laptop.
>> Now, I'm writing this email on 2.6.34.y.)
> 
> But every time you boot 2.6.35 you get errors?  Would it be possible to
> save the console output (netconsole works well)

[...]
device fsid 3247922091b53feb-dcb02f0506fbdc8b devid 1 transid 155748
/dev/mapper/root
EXT3-fs: barriers not enabled
kjournald starting.  Commit interval 5 seconds
EXT3-fs (sda4): warning: maximal mount count reached, running e2fsck is
recommended
EXT3-fs (sda4): using internal journal
EXT3-fs (sda4): recovery complete
EXT3-fs (sda4): mounted filesystem with writeback data mode
btrfs: fail to dirty  inode 9959493 error -28
btrfs: fail to dirty  inode 2987803 error -28
btrfs: fail to dirty  inode 2987803 error -28
btrfs: fail to dirty  inode 8873620 error -28
btrfs: fail to dirty  inode 8873620 error -28
btrfs: fail to dirty  inode 803894 error -28
btrfs: fail to dirty  inode 2988335 error -28
btrfs: fail to dirty  inode 2987971 error -28
btrfs: fail to dirty  inode 2987972 error -28
btrfs: fail to dirty  inode 2988336 error -28
btrfs: fail to dirty  inode 6631 error -28
btrfs: fail to dirty  inode 803896 error -28
btrfs: fail to dirty  inode 6632 error -28
btrfs: fail to dirty  inode 6633 error -28
btrfs: fail to dirty  inode 6634 error -28
[...] (nothing new coming, only error -28)

Apart from that, I had messages that there was no space left on /, but
those were from userspace and not logged via netconsole.

Re: btrfsck segmentation fault + trace

[Yan, Zheng]

On Thu, Jun 17, 2010 at 4:25 AM, John Wyzer <john.wyzer@gmx.de> wrote:
> On 16/06/10 19:29, Chris Mason wrote:
>> On Wed, Jun 16, 2010 at 06:35:25PM +0200, John Wyzer wrote:
>>>> Is 2.6.34 working normally?
>>>
>>> Yes. I can boot with 2.6.34.y and everything works fine. (Actually,
>>> before trying 2.6.45-rc3, I had an uptime of two weeks on this lapt=
op.
>>> Now, I'm writing this email on 2.6.34.y.)
>>
>> But every time you boot 2.6.35 you get errors? =A0Would it be possib=
le to
>> save the console output (netconsole works well)
>
> [...]
> device fsid 3247922091b53feb-dcb02f0506fbdc8b devid 1 transid 155748
> /dev/mapper/root
> EXT3-fs: barriers not enabled
> kjournald starting. =A0Commit interval 5 seconds
> EXT3-fs (sda4): warning: maximal mount count reached, running e2fsck =
is
> recommended
> EXT3-fs (sda4): using internal journal
> EXT3-fs (sda4): recovery complete
> EXT3-fs (sda4): mounted filesystem with writeback data mode
> btrfs: fail to dirty =A0inode 9959493 error -28
> btrfs: fail to dirty =A0inode 2987803 error -28
> btrfs: fail to dirty =A0inode 2987803 error -28
> btrfs: fail to dirty =A0inode 8873620 error -28
> btrfs: fail to dirty =A0inode 8873620 error -28
> btrfs: fail to dirty =A0inode 803894 error -28
> btrfs: fail to dirty =A0inode 2988335 error -28
> btrfs: fail to dirty =A0inode 2987971 error -28
> btrfs: fail to dirty =A0inode 2987972 error -28
> btrfs: fail to dirty =A0inode 2988336 error -28
> btrfs: fail to dirty =A0inode 6631 error -28
> btrfs: fail to dirty =A0inode 803896 error -28
> btrfs: fail to dirty =A0inode 6632 error -28
> btrfs: fail to dirty =A0inode 6633 error -28
> btrfs: fail to dirty =A0inode 6634 error -28
> [...] (nothing new coming, only error -28)
>
> Apart from that, I had messages that there was no space left on /, bu=
t
> those were from userspace and not logged via netconsole.
>

looks like the fs runs out of metadata space. btrfs in 2.6.35 reserves
more metadata space for system use than btrfs in 2.6.34. That's why
these error message only appear on 2.6.35.

Yan, Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: btrfsck segmentation fault + trace

[John Wyzer]

On 17/06/10 00:45, Yan, Zheng wrote:
> On Thu, Jun 17, 2010 at 4:25 AM, John Wyzer <john.wyzer@gmx.de> wrote:
>> btrfs: fail to dirty  inode 6634 error -28
>> [...] (nothing new coming, only error -28)
>>
>> Apart from that, I had messages that there was no space left on /, but
>> those were from userspace and not logged via netconsole.
>>
> 
> looks like the fs runs out of metadata space. btrfs in 2.6.35 reserves
> more metadata space for system use than btrfs in 2.6.34. That's why
> these error message only appear on 2.6.35.

Hmm. I've formatted with -m single because I wanted to avoid running out
of space for metadata. So if there's more then 10% of space left which
is about 40GB that seems like quite a bit of waste... ;-)

I'll stay with 2.6.34 then for the time being.

Thanks.

Re: btrfsck segmentation fault + trace

[Yan, Zheng]

On Thu, Jun 17, 2010 at 7:26 AM, John Wyzer <john.wyzer@gmx.de> wrote:
> On 17/06/10 00:45, Yan, Zheng wrote:
>> On Thu, Jun 17, 2010 at 4:25 AM, John Wyzer <john.wyzer@gmx.de> wrot=
e:
>>> btrfs: fail to dirty =A0inode 6634 error -28
>>> [...] (nothing new coming, only error -28)
>>>
>>> Apart from that, I had messages that there was no space left on /, =
but
>>> those were from userspace and not logged via netconsole.
>>>
>>
>> looks like the fs runs out of metadata space. btrfs in 2.6.35 reserv=
es
>> more metadata space for system use than btrfs in 2.6.34. That's why
>> these error message only appear on 2.6.35.
>
> Hmm. I've formatted with -m single because I wanted to avoid running =
out
> of space for metadata. So if there's more then 10% of space left whic=
h
> is about 40GB that seems like quite a bit of waste... ;-)
>
> I'll stay with 2.6.34 then for the time being.
>

Spaces for data and metadata are separated even the FS was formatted
with -m single.

Yan, Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html