Hi,

We have multiple 22 disk raid 6 arrays using LSI 9280-24i4e raid cards, all using enterprise grade drives. The array is setup with physical drive cache disabled and mounted using inode64 with nobarriers option in Oracle Linux Server release 6.6, kernel version 3.8.13-55.1.5.el6uek.x86_64. We are suffering from corruptions with xfs_repair unable to permanently fix the the issues. Output of dmesg from the latest corruption.

Pid: 5319, comm: glusterfsd Not tainted 3.8.13-55.1.5.el6uek.x86_64 #2
Call Trace:
[<ffffffffa02a3a9f>] xfs_error_report+0x3f/0x50 [xfs]
[<ffffffffa02ed416>] ? xfs_iread_extents+0x86/0x110 [xfs]
[<ffffffffa02a3b0e>] xfs_corruption_error+0x5e/0x90 [xfs]
[<ffffffffa02c7ac8>] xfs_bmap_read_extents+0x3f8/0x450 [xfs]
[<ffffffffa02ed416>] ? xfs_iread_extents+0x86/0x110 [xfs]
[<ffffffffa02eb94e>] ? xfs_iext_realloc_direct+0xae/0x160 [xfs]
[<ffffffffa02ed416>] xfs_iread_extents+0x86/0x110 [xfs]
[<ffffffffa02fb0cc>] ? xfs_trans_free_item_desc+0x3c/0x50 [xfs]
[<ffffffffa02c5df5>] xfs_bmap_last_extent+0x95/0xb0 [xfs]
[<ffffffffa02c5ef6>] xfs_bmap_last_offset+0x66/0xc0 [xfs]
[<ffffffffa02dc123>] ? xfs_dir_lookup+0xc3/0x170 [xfs]
[<ffffffffa02dbc96>] xfs_dir2_isblock+0x26/0x60 [xfs]
[<ffffffffa02dc123>] xfs_dir_lookup+0xc3/0x170 [xfs]
[<ffffffffa02b6497>] xfs_lookup+0x87/0x110 [xfs]
[<ffffffff811a92ed>] ? __d_alloc+0x14d/0x180
[<ffffffffa02aea84>] xfs_vn_lookup+0x54/0xa0 [xfs]
[<ffffffff8119db43>] ? lookup_dcache+0xa3/0xd0
[<ffffffff8119d3ad>] lookup_real+0x1d/0x60
[<ffffffff8119dba8>] __lookup_hash+0x38/0x50
[<ffffffff8119dc0e>] lookup_slow+0x4e/0xc0
[<ffffffff811a1d41>] path_lookupat+0x201/0x780
[<ffffffff811a22f4>] filename_lookup+0x34/0xc0
[<ffffffff811a35b9>] user_path_at_empty+0x59/0xa0
[<ffffffff811a3611>] ? user_path_at+0x11/0x20
[<ffffffff811986c0>] ? vfs_fstatat+0x50/0xb0
[<ffffffff811a3611>] user_path_at+0x11/0x20
[<ffffffff811a3698>] sys_linkat+0x78/0x250
[<ffffffff810e37e6>] ? __audit_syscall_exit+0x216/0x2c0
[<ffffffff815a25d9>] system_call_fastpath+0x16/0x1b
XFS (sdb): Corruption detected. Unmount and run xfs_repair
XFS (sdb): corrupt dinode 6442451040, (btree extents).
ffff88042615d000: 42 4d 41 50 00 00 00 fe ff ff ff ff ff ff ff ff BMAP............
XFS (sdb): Internal error xfs_bmap_read_extents(1) at line 1610 of file fs/xfs/xfs_bmap.c. Caller 0xffffffffa02ed416

What could be causing these issues?

Thanks for your help in advanced.

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________