ext4 error

ext4 error

[Eric Shang]

HI All:
  I meet an ext4 error, following is the error log. After panic, I check the emmc by the tool debufs, the inode 69878 i_nlink is not zero. And this inode don't belong to parent dir 6987, it belong to other file(this inode belong to two files when check by debugfs ncheck), I guess than this inode has beed deleted in memory and already used by other file. But the parent dentry buff_head not flush to emmc. But when lookup this dentry can't find it' in dentry cache, and then lookup_real, read the dentry from emmc, get the file inode which already be deleted. 
  Can any give me some help how to check this issue. My kernel version is 3.18 form Android . I thinks something wrong with dentry cache flush and dirty buff_head flush to emmc. Thanks all!

:Sending Trackball (ACTION_MOVE): 0:(-[44802.841647] EXT4-fs error (device mmcblk0p16): ext4_lookup:1417: inode #69877: comm installd: deleted inode referenced: 69878
5.0[44802.853210] Aborting journal on device mmcblk0p16-8.
,2.[44802.858345] Kernel panic - not syncing: EXT4-fs (device mmcblk0p16): panic forced after error
[44802.858345] 
[44802.868322] CPU: 1 PID: 216 Comm: installd Tainted: P           O   3.18.0-94.02.04 #1
[44802.876223] [<c0016874>] (unwind_backtrace) from [<c00123e8>] (show_stack+0x10/0x14)
[44802.883927] [<c00123e8>] (show_stack) from [<c07af1b0>] (dump_stack+0x70/0xbc)
[44802.891117] [<c07af1b0>] (dump_stack) from [<c07ab328>] (panic+0xa4/0x218)
[44802.897965] [<c07ab328>] (panic) from [<c01c2fa4>] (ext4_handle_error+0x6c/0xa8)
[44802.905330] [<c01c2fa4>] (ext4_handle_error) from [<c01c323c>] (__ext4_error_inode+0x6c/0x114)
[44802.913909] [<c01c323c>] (__ext4_error_inode) from [<c01b2e18>] (ext4_lookup+0xf0/0x148)
[44802.921970] [<c01b2e18>] (ext4_lookup) from [<c0130148>] (lookup_real+0x20/0x50)
[44802.929336] [<c0130148>] (lookup_real) from [<c0130c60>] (__lookup_hash+0x34/0x44)
[44802.936873] [<c0130c60>] (__lookup_hash) from [<c0131f98>] (lookup_slow+0x38/0x9c)
[44802.944414] [<c0131f98>] (lookup_slow) from [<c0132a34>] (path_lookupat+0x228/0x728)
[44802.952127] [<c0132a34>] (path_lookupat) from [<c0132f54>] (filename_lookup+0x20/0xbc)
[44802.960013] [<c0132f54>] (filename_lookup) from [<c0136128>] (user_path_at_empty+0x50/0x78)
[44802.968334] [<c0136128>] (user_path_at_empty) from [<c0136164>] (user_path_at+0x14/0x1c)
[44802.976394] [<c0136164>] (user_path_at) from [<c012c52c>] (vfs_fstatat+0x48/0x8c)
[44802.983847] [<c012c52c>] (vfs_fstatat) from [<c012c820>] (SyS_fstatat64+0x14/0x30)
[44802.991389] [<c012c820>] (SyS_fstatat64) from [<c000eae0>] (ret_fast_syscall+0x0/0x48)

Best Regards
EricShang	

Re: ext4 error

[Toshi Kani]

On Wed, 2016-04-13 at 13:44 +0000, Eric Shang wrote:
> HI All:
>   I meet an ext4 error, following is the error log. After panic, I check the
> emmc by the tool debufs, the inode 69878 i_nlink is not zero. And this inode
> don't belong to parent dir 6987, it belong to other file(this inode belong to
> two files when check by debugfs ncheck), I guess than this inode has beed
> deleted in memory and already used by other file. But the parent dentry
> buff_head not flush to emmc. But when lookup this dentry can't find it' in
> dentry cache, and then lookup_real, read the dentry from emmc, get the file
> inode which already be deleted.
>   Can any give me some help how to check this issue. My kernel version is 3.18
> form Android . I thinks something wrong with dentry cache flush and dirty
> buff_head flush to emmc. Thanks all!

Are you somehow using DAX and my dax_get_unmapped_area patches in your Android
3.18 kernel? I doubt this is the case, but I thought I should check since you
reported it to me and Matthew with the same cc list.

If you are simply using a distributed Android kernel, you should report it to
Android email list or bug tracker.

-Toshi

答复: ext4 error

[Eric Shang]

Hi Toshi:
	Thanks! I think this not relate to DAX and dax_get_unmapped_area, I am a junior kernel maintainer for our company. This issue bother me long time, looks like kernel ext4 issue or our emmc driver issue, I will dig it out, Thanks!

Best Regards
EricShang


-----邮件原件-----
发件人: Toshi Kani [mailto:toshi.kani@hpe.com] 
发送时间: 2016年4月13日 22:10
收件人: Eric Shang; Matthew Wilcox
抄送: akpm@linux-foundation.org; dan.j.williams@intel.com; viro@zeniv.linux.org.uk; ross.zwisler@linux.intel.com; kirill.shutemov@linux.intel.com; david@fromorbit.com; jack@suse.cz; tytso@mit.edu; adilger.kernel@dilger.ca; linux-nvdimm@lists.01.org; linux-fsdevel@vger.kernel.org; linux-ext4@vger.kernel.org; xfs@oss.sgi.com; linux-kernel@vger.kernel.org
主题: Re: ext4 error

On Wed, 2016-04-13 at 13:44 +0000, Eric Shang wrote:
> HI All:
>   I meet an ext4 error, following is the error log. After panic, I 
> check the emmc by the tool debufs, the inode 69878 i_nlink is not 
> zero. And this inode don't belong to parent dir 6987, it belong to 
> other file(this inode belong to two files when check by debugfs 
> ncheck), I guess than this inode has beed deleted in memory and 
> already used by other file. But the parent dentry buff_head not flush 
> to emmc. But when lookup this dentry can't find it' in dentry cache, 
> and then lookup_real, read the dentry from emmc, get the file inode which already be deleted.
>   Can any give me some help how to check this issue. My kernel version 
> is 3.18 form Android . I thinks something wrong with dentry cache 
> flush and dirty buff_head flush to emmc. Thanks all!

Are you somehow using DAX and my dax_get_unmapped_area patches in your Android
3.18 kernel? I doubt this is the case, but I thought I should check since you reported it to me and Matthew with the same cc list.

If you are simply using a distributed Android kernel, you should report it to Android email list or bug tracker.

-Toshi

Re: ext4 error

[Theodore Ts'o]

On Wed, Apr 13, 2016 at 01:44:55PM +0000, Eric Shang wrote:
> HI All:
>   I meet an ext4 error, following is the error log. After panic, I check the emmc by the tool debufs, the inode 69878 i_nlink is not zero. And this inode don't belong to parent dir 6987, it belong to other file(this inode belong to two files when check by debugfs ncheck), I guess than this inode has beed deleted in memory and already used by other file. But the parent dentry buff_head not flush to emmc. But when lookup this dentry can't find it' in dentry cache, and then lookup_real, read the dentry from emmc, get the file inode which already be deleted. 
>   Can any give me some help how to check this issue. My kernel version is 3.18 form Android . I thinks something wrong with dentry cache flush and dirty buff_head flush to emmc. Thanks all!

If I had to guess, this was caused starting with a corrupted file
system, where the inode allocation bitmap showed that an inode which
was in use by the file system, was erroneously showing it as free.
This allowed it to be allocated for use in a second file (which would
have wiped out the contents for the original file stored at that
inode).  Later on, the file was deleted via either the older or newer
pathname, which dropped the ref count to zero, and then an access via
the other pathname would have resulted in this error.

After the panic, the on-disk data structures wouldn't have been
updated from whatever the in-memory data structures might have been
("Kernel panic - not syncing").  So what you see from using debugfs
after the crash might not be represenatative of what you saw before
the crash.

I'm not sure there's much debugging that can be done, because there
are any number of sources for the original corruption.  It could be
caused by a hardware issue in the flash or the memory, or it could be
caused by a wild pointer corrupting a disk buffer, etc. etc.  The
panic won't result in a useful stack trace because that's when the
problem was *noticed*.  But that's very different from where the file
system corruption was *introduced*.

If you can reliably reproduce this sort of failure, then it becomes
possible to try to track it down.  But if it's a one-off event,
there's not much anyone can do.

Best regards,

						- Ted