mutt segfault with ext3 & 1k blocks & htree in 2.6

mutt segfault with ext3 & 1k blocks & htree in 2.6

[Mike Fedyk]

Hi,

I have just converted my 25GB / partition from reiserfs to ext3 with 1k
blocks, and now mutt is segfaulting periodocally.

I suspect it is htree, because I left four mutt processes running last
night, and two of them segfaulted.

Interestingly enough, it happened about 1 minute apart, so they might have
checked one of the large maildir folders and that could have caused the
problem, except that the other two mutt processes should have checked the
same folders, and they didn't crash.

I have full strace output of each mutt process up until the segfault in two
cases, and up until strace was stopped in the third case.

Please let me know what more I can do to help track this down.

I have tried this with:
vmlinuz-2.6.0-test3-mm3
vmlinuz-2.6.0-test4-mm1

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Mike Fedyk]

On Fri, Aug 29, 2003 at 10:24:51AM -0700, Mike Fedyk wrote:
> Hi,
> 
> I have just converted my 25GB / partition from reiserfs to ext3 with 1k
> blocks, and now mutt is segfaulting periodocally.
> 
> I suspect it is htree, because I left four mutt processes running last
> night, and two of them segfaulted.
> 
> Interestingly enough, it happened about 1 minute apart, so they might have
> checked one of the large maildir folders and that could have caused the
> problem, except that the other two mutt processes should have checked the
> same folders, and they didn't crash.
> 
> I have full strace output of each mutt process up until the segfault in two
> cases, and up until strace was stopped in the third case.
> 
> Please let me know what more I can do to help track this down.
> 
> I have tried this with:
> vmlinuz-2.6.0-test3-mm3
> vmlinuz-2.6.0-test4-mm1
> 

How do I:

 o Find out that a directory is using htree?
 o Disable htree on my /?  (tune2fs -O ^dir_index), but then how do I get my
   directories back to non-htree without running fsck from a rescue CD?

Thanks,

Mike

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Andy Isaacson]

On Fri, Aug 29, 2003 at 10:24:51AM -0700, Mike Fedyk wrote:
> I have just converted my 25GB / partition from reiserfs to ext3 with 1k
> blocks, and now mutt is segfaulting periodocally.
[snip]
> I have full strace output of each mutt process up until the segfault in two
> cases, and up until strace was stopped in the third case.

The obvious request is "turn on core dumps and get a backtrace".

-andy

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Mike Fedyk]

On Fri, Aug 29, 2003 at 01:33:23PM -0500, Andy Isaacson wrote:
> On Fri, Aug 29, 2003 at 10:24:51AM -0700, Mike Fedyk wrote:
> > I have just converted my 25GB / partition from reiserfs to ext3 with 1k
> > blocks, and now mutt is segfaulting periodocally.
> [snip]
> > I have full strace output of each mutt process up until the segfault in two
> > cases, and up until strace was stopped in the third case.
> 
> The obvious request is "turn on core dumps and get a backtrace".

The problem only shows on 2.6, and it works perfectly on 2.4.  I don't think
it's an app issue.  Also, Debian doesn't compile in the debug symbols, so
the backtrace is not very much use.  (though, I can show the output if you'd
like, just give me another day for it to segfault again.)

Mike

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Andreas Dilger]

On Aug 29, 2003  11:09 -0700, Mike Fedyk wrote:
> On Fri, Aug 29, 2003 at 10:24:51AM -0700, Mike Fedyk wrote:
>  o Find out that a directory is using htree?

"lsattr <dir>" will show it.  Note that it will only ever be set on
directories that are larger than a single disk block.

# lsattr -d d1
----------I-- d1

>  o Disable htree on my /?  (tune2fs -O ^dir_index), but then how do I get
>    my directories back to non-htree without running fsck from a rescue CD?

That's the great thing about htree - you don't need to do anything to turn
it off.  The on-disk format is exactly the same as without htree, and the
first time you modify the directory it will clear the per-directory htree
flag.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Mike Fedyk]

On Sat, Aug 30, 2003 at 01:14:21PM -0600, Andreas Dilger wrote:
> On Aug 29, 2003  11:09 -0700, Mike Fedyk wrote:
> > On Fri, Aug 29, 2003 at 10:24:51AM -0700, Mike Fedyk wrote:
> >  o Find out that a directory is using htree?
> 
> "lsattr <dir>" will show it.  Note that it will only ever be set on
> directories that are larger than a single disk block.
> 
> # lsattr -d d1
> ----------I-- d1
> 

Ok, now I only have htree enabled on one of my maildir folders.

> >  o Disable htree on my /?  (tune2fs -O ^dir_index), but then how do I get
> >    my directories back to non-htree without running fsck from a rescue CD?
> 
> That's the great thing about htree - you don't need to do anything to turn
> it off.  The on-disk format is exactly the same as without htree, and the
> first time you modify the directory it will clear the per-directory htree
> flag.

I'll do more testing to see if it fails only on that folder now.

But how do I re-enable htree on the directories (besides an fsck -D) in a
live system?

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Andreas Dilger]

On Aug 30, 2003  16:58 -0700, Mike Fedyk wrote:
> On Sat, Aug 30, 2003 at 01:14:21PM -0600, Andreas Dilger wrote:
> > On Aug 29, 2003  11:09 -0700, Mike Fedyk wrote:
> > > On Fri, Aug 29, 2003 at 10:24:51AM -0700, Mike Fedyk wrote:
> > >  o Find out that a directory is using htree?
> > 
> > "lsattr <dir>" will show it.  Note that it will only ever be set on
> > directories that are larger than a single disk block.
> > 
> > # lsattr -d d1
> > ----------I-- d1
> > 
> 
> Ok, now I only have htree enabled on one of my maildir folders.
> 
> > >  o Disable htree on my /?  (tune2fs -O ^dir_index), but then how do I get
> > >    my directories back to non-htree without running fsck from a rescue CD?
> > 
> > That's the great thing about htree - you don't need to do anything to turn
> > it off.  The on-disk format is exactly the same as without htree, and the
> > first time you modify the directory it will clear the per-directory htree
> > flag.
> 
> I'll do more testing to see if it fails only on that folder now.
> 
> But how do I re-enable htree on the directories (besides an fsck -D) in a
> live system?

You need to re-enable the dir_index feature, and then for directories which
are larger than a block in size you need something like:

	mkdir new_dir
	mv old_dir/* new_dir
	rmdir old_dir
	mv new_dir old_dir

The new directory will have htree enabled because it started out at 1 block
in size.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Mike Fedyk]

On Sun, Aug 31, 2003 at 04:44:48PM -0600, Andreas Dilger wrote:
> On Aug 30, 2003  16:58 -0700, Mike Fedyk wrote:
> > But how do I re-enable htree on the directories (besides an fsck -D) in a
> > live system?
> 
> You need to re-enable the dir_index feature, and then for directories which
> are larger than a block in size you need something like:
> 
> 	mkdir new_dir
> 	mv old_dir/* new_dir
> 	rmdir old_dir
> 	mv new_dir old_dir
> 
> The new directory will have htree enabled because it started out at 1 block
> in size.

Ok I ended up doing this after a little thought.  Thanks.

But I am seeing segfaults in mutt under 2.6 ext3 with 1k blocks, that I
don't see in 2.4, and didn't see under reiserfs.  I can try with 4k blocks,
if you'd like, is there anything I can do to capture more information that
could be happening to cause this?

And now mutt is segfaulting on non-htree directories too.

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Andreas Dilger]

On Sep 01, 2003  13:27 -0700, Mike Fedyk wrote:
> On Sun, Aug 31, 2003 at 04:44:48PM -0600, Andreas Dilger wrote:
> > On Aug 30, 2003  16:58 -0700, Mike Fedyk wrote:
> > > But how do I re-enable htree on the directories (besides an fsck -D) in a
> > > live system?
> > 
> > You need to re-enable the dir_index feature, and then for directories which
> > are larger than a block in size you need something like:
> > 
> > 	mkdir new_dir
> > 	mv old_dir/* new_dir
> > 	rmdir old_dir
> > 	mv new_dir old_dir
> > 
> > The new directory will have htree enabled because it started out at 1 block
> > in size.
> 
> Ok I ended up doing this after a little thought.  Thanks.
> 
> But I am seeing segfaults in mutt under 2.6 ext3 with 1k blocks, that I
> don't see in 2.4, and didn't see under reiserfs.  I can try with 4k blocks,
> if you'd like, is there anything I can do to capture more information that
> could be happening to cause this?

Normally an application segfault is really caused by a kernel OOPS, so if
you look into your syslog file or dmesg output you should see an oops.

> And now mutt is segfaulting on non-htree directories too.

I couldn't comment on that, but either the directories are somehow corrupted
(e2fsck will know), or the problem is related either to 1kB blocks or not
related to the filesystem at all.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
On Sep 01, 2003  13:27 -0700, Mike Fedyk wrote:
> On Sun, Aug 31, 2003 at 04:44:48PM -0600, Andreas Dilger wrote:
> > On Aug 30, 2003  16:58 -0700, Mike Fedyk wrote:
> > > But how do I re-enable htree on the directories (besides an fsck -D) in a
> > > live system?
> > 
> > You need to re-enable the dir_index feature, and then for directories which
> > are larger than a block in size you need something like:
> > 
> > 	mkdir new_dir
> > 	mv old_dir/* new_dir
> > 	rmdir old_dir
> > 	mv new_dir old_dir
> > 
> > The new directory will have htree enabled because it started out at 1 block
> > in size.
> 
> Ok I ended up doing this after a little thought.  Thanks.
> 
> But I am seeing segfaults in mutt under 2.6 ext3 with 1k blocks, that I
> don't see in 2.4, and didn't see under reiserfs.  I can try with 4k blocks,
> if you'd like, is there anything I can do to capture more information that
> could be happening to cause this?

Normally an application segfault is really caused by a kernel OOPS, so if
you look into your syslog file or dmesg output you should see an oops.

> And now mutt is segfaulting on non-htree directories too.

I couldn't comment on that, but either the directories are somehow corrupted
(e2fsck will know), or the problem is related either to 1kB blocks or not
related to the filesystem at all.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: mutt segfault with ext3 & 1k blocks & htree in 2.6

[Mike Fedyk]

On Tue, Sep 02, 2003 at 10:09:27AM -0600, Andreas Dilger wrote:
> Normally an application segfault is really caused by a kernel OOPS, so if
> you look into your syslog file or dmesg output you should see an oops.
> 

No, I'm not seeing any oopses on this machine at all right now (not that I
couldn't cause a couple at will, but those are reported with varying degrees
of success on getting fixes)

> > And now mutt is segfaulting on non-htree directories too.
> 
> I couldn't comment on that, but either the directories are somehow corrupted
> (e2fsck will know), or the problem is related either to 1kB blocks or not
> related to the filesystem at all.

Ok, I will convert my ext3 (cp twice) to 4k blocks, and try to reproduce.
Hopefully I didn't hit a library update (in debian a mixed debian testing
/ unstable system) that caused this problem.